Sorting by primary key before inserting data into DB is fixed.
Earlier we were sorting input slice of BucketBandwidthRollup but then
we were putting all entries into map to rollup input data. Iteration
over map with a range loop doesn't guarantee any specific order so we
were loosing sorted order when we were creating with this map slices to
use with DB insert.
New code is also using map but when map is full its sorting map keys
separately and iterates over them to get data from map.
https://github.com/storj/storj/issues/5332
Change-Id: I5bf09489b0eecb6858bf854ab387b660124bf53f
The previous test relied on timing, but instead let's try dialing
the server and see whether we can do something with the connection.
We probably should test all the supported protocols instead of just tcp.
Change-Id: I9217494859faea0a7b93515aad706da4fdd8a140
`storage.KeyValueStore` requires ordered iteration, which redis
doesn't support natively. This would require loading all the keys
into memory and then processing them, rather than iterating over them
one-by-one.
This adds a temporary `IterateUnordered` to handle the migrations
more gracefully.
Change-Id: I55b763500523077c7ab8fdfad175c32cc7788e47
Allow user to create bucket without a passphrase if project level passphrase was not set.
Issue:
https://github.com/storj/storj/issues/5521
Change-Id: Ifc4a6724229ce0708db720edb2f8953098e346ed
uplinks currently get the node's certificate chain over TLS. once Noise
is in use, uplinks will no longer be able to do this. we should start
having the upload request return the certificate chain in the same
release that starts supporting noise.
Change-Id: I619b23cb8e25691bcc62d760f884403a4ccd64a0
File paths may contain characters that have special meaning when placed
in URLs. For example, a folder name may contain a pound symbol (#)
which is the URL fragment delimiter. This causes the characters that
succeed this symbol to not be considered part of the primary resource
identifier as they should be, resulting in navigation errors.
This change resolves this issue.
Resolves#5522
Change-Id: I59972321795375ec5981c3e9c505e35f998022d6
`overlay.(*Service).UpdateReputation()` takes a "reputationChanges"
parameter, a slice of node events indicating whether we think the node's
disqualification or suspension status is changing. This is necessary so
that the overlay service can notify the nodeevents DB about these
changes.
In several cases, however, this list of events is not constructed
correctly, because of missing information about the previous state.
In most cases, this is because the node was offline, and the order limit
creation functions (which usually obtain and return the prior reputation
status) ignored that node.
This change makes it so that all callers to
`overlay.(*Service).UpdateReputation()` can be expected to provide a
correct list of change events (as correct as feasible, given that we
can't lock the node's information in the database during the entire
operation).
It ended up that there was only one caller we needed to worry about, and
that was reputation.(*Service).ApplyAudit(). So the bulk of this change
is teaching that function how to recognize when the prior reputation
status was not filled in, and fill it in.
Refs: https://github.com/storj/storj/issues/5464
Change-Id: I52ce385fc9c0ce3b283b998d517998e7f4ec8792
Affected packages admin,attribution,console,metainfo,satellitedb,web,payments
This change removes the satellite/rewards package and its related usages.
It removes references to APIKeyInfo/PartnerID, Project/PartnerID
and User/PartnerID.
Issue: https://github.com/storj/storj/issues/5432
Change-Id: Ieaa352ee848db45e94f85556febdbcf1444d8c3e
This tool is being removed because it has served its purpose and was blocking another removal from being verified.
Change-Id: Ie888aa7ae1b153a34210af3a5d5a3682b381ba82
A user on the forum was seeing the error "bad message", which was not
very helpful. This case from the ext4 filesystem using the code EBADMSG
to indicate it detected an invalid CRC, suggesting disk corruption.
This change adds some explanatory information about probable disk
corruption to all errors coming from the (*blobInfo).Stat() call, which
is where storagenode fs corruption problems will usually manifest.
Refs: https://github.com/storj/storj/issues/5375
Change-Id: I87f4a800236050415c4191ef1a0fc952f9def315
Calling stat() (really, lstat()) on every file during a directory walk
is the step that takes up the most time. Furthermore, not all directory
walk uses _need_ to have a stat done on every file. Therefore, in this
commit we avoid doing the stat at the lowest level of
walkNamespaceInPath. The stat will still be done when it is requested,
with the Stat() method on the blobInfo object.
The major upside of this is that we can avoid the stat call on most
files during a Retain operation. This should speed up garbage collection
considerably.
The major downside is that walkNamespaceInPath will no longer
automatically skip over directories that are named like blob files, or
blob files which are deleted between readdir() and stat(). Callers to
walkNamespaceInPath and its variants (WalkNamespace,
WalkSatellitePieces, etc) are now expected to handle these cases
individually.
Thanks to forum member Toyoo for the insight that this would speed up
garbage collection.
Refs: https://github.com/storj/storj/issues/5454
Change-Id: I72930573d58928fa25057ed89cd4ec474b884199
Add migration tool (and test) to update salt column in projects table
with the SHA-256 hash of the project ID when null
Issue https://github.com/storj/storj-private/issues/66
Change-Id: Ib8d484ac8d6ee25859064d803e2ac8fb46b45921
Our DB support in storj/private was updated to enable basic context
support for executing SQL queries. This change requires some small
adjustments as not all parts were working correctly.
storj/private commit with change:
4bc77107b7acfcc2f7ad65796d5dd3d7c64801e4
Change-Id: I64d7ed92788ea0920d12cecd1aa0e414720e9b9c
Show create/enter passphrase modal after login for new project level passphrase flow.
Also fixed buckets view mounted hook to load create bucket modal instead of old flow.
Issue:
https://github.com/storj/storj/issues/5510
Change-Id: If9ea70faaa2987f336d72d55a6ed2bbd02ced592
This removes the indirect dependency to go.opentelemetry.io/otel, which
causes a dependency conflict in storj.io/ipfs-go-ds-storj.
Change-Id: I28c1e3637bb23f7949988511051d39c08b3fc852
This removes the indirect dependency to go.opentelemetry.io/otel, which
causes a dependency conflict in storj.io/ipfs-go-ds-storj.
Change-Id: I28c1e3637bb23f7949988511051d39c08b3fc852
gob package is not stable across Go version, let's switch to protobuf
for encoding these. We still need backwards compatibility for the
moment.
Change-Id: If1da50658ab39a75d1b2b1f988356b56347cac14
Fixed notifications link to toggle manage passphrase modal instead of redirecting to buckets screen.
Issue:
https://github.com/storj/storj/issues/5452
Change-Id: I84ce67da7445c6316eb0658606c0129bb4af680e
Add a flag to enable/disable analytics so uplink can be run
non-interactively. Also when run non-interactively for the first time
it will not error any more but instead default to disable analytics.
Part of https://github.com/storj/storj/issues/5126
Change-Id: I07ac8a040664334efcb4e2536f26c330c1751a6f
When running pay invoices command skip invoices that has due date set, so
customers can still pay it themselves until due date.
https://github.com/storj/storj/issues/5453
Change-Id: I8e557062491ab0c8246b28bc5ca57e845eb32e29
- Adds "Remote Address" field to all INFO logs related to GET,
PUT, and DELETE requests
- Adds Offset and Size fields to all info logs related to GET
requests
Resolves https://github.com/storj/storj/issues/5404
Change-Id: I5dab1867619385362e5f1e0455dfab17d295a37a
Add a docker image for uplink-cli and push it to docker hub.
We used to have this before the change to uplinkng. I'm not
sure if the pushing works, we'll see after merge.
To test, build an image with `make uplink-image`, read the tag from the
output and run normal uplink-cli commands using
`docker run -it storjlabs/uplink:df9bbceca-uplink-docker-go1.18.8-amd64 [command]`
Part of https://github.com/storj/uplink/issues/109
Change-Id: I8a10aab2b778951ff42a22ba2f252c581eb66b65
We were reading in a segment's stream ID and position, and assuming that
was enough for the downloader. But of course, the downloader needs
AliasPieces filled in. So now we request each segment record from the
metabase and fill in the VerifySegment records entirely.
Change-Id: If85236388eb99a65e2cb739aa976bd49ee2b2c89
This is automated test around metabase tests. It's detecting queries
which were performing full table scan during test execution.
After merging this and checking that this is not problematic in any way
we will enable this also for testplanet tests.
One query was adjusted to avoid full table scan and pass new check.
https://github.com/storj/storj/issues/5471
Change-Id: I2d176f0c25deda801e8a731b75954f83d18cc1ce
When there is no wallet in the database for a particular customer
return 404 http response status code instread of internal server error.
Change web/satellite payments API to return empty wallet on 404 response
code instead of throwing an error.
Change-Id: Ib44914f9ed002382258968fb81846f2b97dee0fe
Fixed Vuex state clearing on session timeout.
Now state is being cleared right after redirect to login and before logout API call.
Change-Id: I3cddb85735899f0913a273a49335730a4097435e
