We will introduce new logic for creating new objects (BeginObject).
Instead of using single version internally (1) we will be selecting first
available version during object creation. Because we need to be sure
that everything is wired up correctly we need a feature flag to be
able to control if new feature is enabled.
Change-Id: If0f8496397130811f43bf9db9fdcc2b30cd2e4ca
Assert that listing pending objects works with our new object consistency
approach.
Half of https://github.com/storj/storj/issues/4868
Change-Id: Ic7bf3b20db57e64853d0464d7dc0da5441efd56f
2.7 natively supports the Composition API, which makes transitioning to
Vue 3 easier to do step-by-step.
see: https://github.com/storj/storj/issues/5145
Change-Id: I34e13b4e4c07176f73a89ef00e3dcd6767694554
Basically, I reverted this change with some modifications.
https://review.dev.storj.io/c/storj/storj/+/6731
We don't use signed requests anymore because its usage was not really affective and caused some UX problems like too long video loading/buffering.
We don't use presigned URLs as well because they are not really compatible (I think) with video streaming. So long videos (> 1 min long) constantly failed to buffer.
Instead of presigned URLs we just use STORJ linksharing. We generate new access grant which is valid for one day to render preview and object map.
Issue:
https://github.com/storj/storj/issues/5025
Change-Id: I0313f2c6b78811f1058c25fdf5b94dc5d563c15d
* service was running wrong RunOnce method
* after doing integration with GC sender we concluded that we don't
need special flag "gc-sender" to be uploaded as its safe to consume
partial results by GC senger. This part was removed.
* prefix format for moving data after error was unified with GC
sender
https: //github.com/storj/team-metainfo/issues/120
Change-Id: I204b696b9c2def4874ad1d17d0e84231cc98d583
Implement a new service to read retain filter from a bucket and
send them out to storagenodes.
This allows the retain filters to be generated by a separate command on
a backup of the database.
Paralellism (setting ConcurrentSends) and end-to-end garbage collection
tests will be restored in a subsequent commit.
Solves https://github.com/storj/team-metainfo/issues/121
Change-Id: Iaf8a33fbf6987676cc3cf74a18a8078916fe673d
Current metainfo.ListObjects implementation is using metabase iterator to list objects.
In the non-recursive case, it used to retrieve all the corresponding rows and then discarded the entries that did not fit the listing request.
This can lead in some edge cases (each prefix contains more than batchsize objects/sub-prefixes) to make unecessary calls to the db.
This change defines the metabase.ListObjects and aims at retrieving only prefixes (but not objects under it) and objects by modifying the SQL query.
In this version, it is not optimized on the database side. Cockroach will still have to go through all rows under a prefix, so there is still room for improvement.
metainfo.ListObjects is not currently using this method as we would like to assess its performance on the QA satellite first.
Fixes https://github.com/storj/storj/issues/5088
Change-Id: Ied3a9210871871d9d4a3096888d3e40c2dceed61
Fixed folder upload for object browser.
Seems like the issue appeared because user has to cofirm folder upload in browser alert message now.
So the problem is that folder upload input is getting unloaded before user approves folder upload.
Change-Id: I5d8377106e5aed74d7c38b2dc88f75370c828d62
Rather than using Invoice Items to account for storjscan token payments, credit notes will be used and applied to the users finalized invoice. This credit note will reduce the amount due of the users invoice based on the amount of storj token balance the user has on the satellite. Applying credit notes to a finalized invoice also requires that the invoice not be automatically paid when finalized. Therefore, a new command (pay-invoices) was added to initiate payment for users invoices.
Change-Id: Ie539375a10e842e3cb64bf0140834bbab0774f54
Doing some cleanup in "scripts" folder. All integration like tests are
moved under "test" directory (integration, bc, redis) and bash scripts
are adjusted to reflect new location.
As an addition "scripts/install-awscli.sh" was deleted as it was not
used.
Change-Id: I152905c4258f471a71f2d0e8731d91bb075e99c1
This change resolves an issue where an error would be shown to the
user due to inappropriate removal of the Stripe card input element
upon attempted submission of credit card info.
Change-Id: I479ea58bfd2a457da5cb6a9c7bff476269c63b23
They are needed for segment-verify tool.
Also rename some of the conversion methods to make clear,
which of them have side-effects.
Change-Id: Ie9a0952548e9ed5068c7a30c2fd2134b07139bca
The collector tries deleting a piece over and over again, though
the piece does not exist on the storagenode's filesystem.
We need to delete the piece info from the expired db if the
targeted file does not exist.
This does not resolve the base problem of why the file
is deleted before the collector tries deleting it.
This change deletes the piece info from the expired db
if the file does not exist, since we're already trying
to delete that piece anyway.
Closes https://github.com/storj/storj/issues/4192
Change-Id: If659185ca14f1cb29fd3c4237374df6fcd535df8
Bloom filter CreationDate is used to avoid deleting pieces that
where not processed by GC. Every piece created after that timestamp
won't be deleted. Current GC process is taking CreationDate as a
beginning of bloom filter creation. This is appraoch allows to avoid
issues with inconsistent view on DB as currently we are using live DB
to create bloom filters.
With appraoch were we will be using DB snaphot with segment loop
we can get CreationDate from latest created segment in DB. Every piece
created after latest created segment won't be touched by GC on storage
node.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: I6aaf64948ab7f60cfea62195689ad77c25ea772e
This adds parts for:
1. iterating over the segments
2. using an interface for writing the segments
3. stubs for handling deleted segments
Change-Id: I76a17cac6deb0b6c042a8ab7c4155a890db9da84
Update storjscan API payment type to use currency.Amount for token and
usd values.
Update storjscan version for testsuite intgration tests.
Change-Id: Ie4ba7277b25d0d297a0130a8b885d6d38ff5eea4
If source and destination are the same, do nothing and return the original object.
Later, we will have to handle the case when metadata is changed, but for now it's not possible through libuplink.
An issue has been created for this (https://github.com/storj/storj/issues/5168)
Change-Id: I91cf48afeec498d3b2c219fa0d3baf2163cff384
This patch fix the beavior of the distributed tracing reporter.
1. For developer build we don't append the date
* We don't need to separate service instances in jaeger (search by trace ID)
* It's usually 0000-00-000 anyway as release.sh is not used for dev builds
2. Tracing ID MUST be unique
* Instead of trusting the user to set a unique value (how can they do it?), we generate a random number
3. To make it possible to find the trace, there is a new flag to print out the generated tracing ID
4. Monkit `remoteTrace` call is replaced with normal monkit Task.
* remoteTrace call assumes that we have a parent span in an other service (which is already sent to the server)
* Here we must send out the parent span, as this is the beginning of the trace
5. We properly close the Jaeger UDP collector, and we wait until remaining messages are sent out
Change-Id: Iabf5abf25f4f20881188f88edcbadca95ac74927
By including NodeURL in the OIDC well-known configuration, we're
able to discover the NodeURL for a given HTTP address without
any client side assumptions or needing to make a drpc to discover
this. Instead, it's included in a call that is already made by an
OIDC/OAuth enabled client.
Change-Id: If00f31665ca69b1f522e26fec825b29ad03fe7f9
Fixed X-icon positioning for add card container on new billing screen.
Issue:
https://github.com/storj/storj/issues/5120
Change-Id: I9c2a62a8ff9a268671d798d1568b07d54c6282a6
Implements creating roughly load-balanced set of batched
that can be used to make multiple requests.
Change-Id: I349b276176dcb8ba9163e7e06a94509d73fa5ddc
We need a method for getting a list of segments from the metabase,
without converting the aliases and omitting all inline segments.
Change-Id: I26d919c675fc285ab03a35b327edd9b5c8bbe4b0
Main issue with those tests was that for case where all objects were
uploaded at once (case "some nodes down" and "all nodes down").
Because all objects had the same name while upload each new object
was overwriting existing object. Because of that instead had several
objects to delete by test explicitly we had just 1. Test were not
failing because while overwriting existing object we were deleting it
but it was not what this test should do.
Change-Id: I602116f00be66589c7c0e68fe28c25e5c03e6b5d
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket.
This change add main logic to new service. After collecting all bloom
filters with segment loop and piece tracker all filters are marshaled
and packed into zip files. Each zip contains up to "ZipBatchSize" bloom
filters and it's uploaded to specified in configuration bucket.
All uploaded objects have specified expiration time to not delete them
manually.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: I2b6bc02a7dd7c3a639e75810fd013ae4afdc80a2
We plan to replace metabase.BeginObjectExactVersion usage in
metainfo.BeginObject with metabase.BeginObjectNextVersion. To make this
switch as simple a possible would be nice to have the same results for
both methods. This change is extending return value for
BeginObjectNextVersion to whole object struct. Tests were also adjusted
to be more like metabase.BeginObjectExactVersion tests.
Part of https://github.com/storj/storj/issues/4871
Change-Id: I4db99d74af07e5a73757b55233e0bbdc7b99d565
existing adaptations fixed
added new adaptation media queries
modals related to setting page fixed on larger screens
cursor changed to pointer for passphrase type selection
Change-Id: Iee3dd141c2beea6951ff43b31506e709fb1d6b9d
This change implements a requirement that all new passwords must be
no longer than 128 characters.
Change-Id: I7a8f23c87190d465eed59b3e0627ccd32d465bb9
Remove pkg satellite/payments/monetary as it moved to storj.io/common.
Update all code pkg references from monetary to common/currency.
Change-Id: If2519f4c80cf315a9299e6521a6b9bbc6c399156
This change modifies the new project dashboard to accommodate
small screen sizes.
Resolves https://github.com/storj/storj/issues/5036
Change-Id: I27a8acd144a24f21f0fa43808640ba5bca6b49a8
v-table visual bug with name fixed
v-header search position fixed
v-search expands 100% on smaller screens
Change-Id: Iadd6ecea4fba3d39af5b4ea48705f2e199810080
This change makes actions to hide the bucket guide much clearer by using only one "OK" action button.
see: https://github.com/storj/storj/issues/5094
Change-Id: I3de8c1e21a5ec5dbcd91e69c1579fac7f46d39ca
Our Test Versions still requires 1.16 to be compatible with our oldest
uplink versions. These changes make the code compile with 1.16.
Also, it makes go generate work in private/apigen/example.
Change-Id: Ib2f7493941a16f361328fe01d2be293f26123719
