With this change we are replacing parsing code with existing go-redis
util.
We also switch redis client to version 9.
Change-Id: Ie4a651e3ae6960e68958c690873925d319b70e10
to support TCP_FAST_OPEN, we're considering just using
two TCP connections in parallel per request, one with
and one without. this allows us to safely fire both
concurrently without stressing out the node too much.
see https://review.dev.storj.io/c/storj/storj/+/9933
Change-Id: I9aa8a0252350db5ace04ee125bfe469203e980ec
Option added to `uplink access setup` and `uplink access create`
commands to disable object key encryption.
Related to https://github.com/storj/storj/issues/5678
Change-Id: I4789a94143742ff4b232fd60decc029ad2883c2a
* testsuite/playwright: copys files to storj repo
This PR copies all the playwright ui tests to the storj repository.
* CI: Jenkinsfile.ui
Jenkinsfile.ui builds all necessary items for running storj binaries, installs all items for playwright ui tests and executes the test. Note that we aren't running regular unit tests and we are not linting the code as this is occuring in the public instance. Furthermore, we still need to implement the starting of storj services as well as typescript linters for the playwright tests.
* testsuite: playwright ui tests
This PR copies all playwright ui tests files from the qa repo.
* testsuite: playwright ui tests
This PR copies all playwright ui tests files from the qa repo.
This code is essentially replacement for eestream.CalcPieceSize. To call
eestream.CalcPieceSize we need eestream.RedundancyStrategy which is not
trivial to get as it requires infectious.FEC. For example infectious.FEC
creation is visible on GE loop observer CPU profile because we were
doing this for each segment in DB.
New method was added to storj.Redundancy and here we are just wiring it
with metabase Segment.
BenchmarkSegmentPieceSize
BenchmarkSegmentPieceSize/eestream.CalcPieceSize
BenchmarkSegmentPieceSize/eestream.CalcPieceSize-8 5822 189189 ns/op 9776 B/op 8 allocs/op
BenchmarkSegmentPieceSize/segment.PieceSize
BenchmarkSegmentPieceSize/segment.PieceSize-8 94721329 11.49 ns/op 0 B/op 0 allocs/op
Change-Id: I5a8b4237aedd1424c54ed0af448061a236b00295
v72.90.0 grants access to InvoiceParams.PendingInvoiceItemsBehavior
which allows the caller to specify whether the invoice should apply all
pending invoice items.
Change-Id: I7c8295d38791ed7daa3d5b1fce15438eb42a962f
this change uses the new storj/common noise helpers, which:
* add a security fix (require an expected node id for validating
noise key attestations)
* stops doing an unnecessary order signature validation (it's
already been done inside of PutPiece)
* removes some duplicate code
Change-Id: I5e67a08ff216cd9c5b0b82e40b4d9de664b6b0fc
This bumps common, such that things build with Go 1.20.
Also, adds `go vet` checks for testsuite/storjscan and testsuite/ui.
The latest golang.org/x/bcrypt has a check that the new password is less
than 72 bytes, because bcrypt silently discarded them. This means our
own password validation has the same limitation. Old passwords should
still work fine.
Change-Id: Ibb8735b15eeb91460145906b81ae4e365e9ac418
Our DB support in storj/private was updated to enable basic context
support for executing SQL queries. This change requires some small
adjustments as not all parts were working correctly.
storj/private commit with change:
4bc77107b7acfcc2f7ad65796d5dd3d7c64801e4
Change-Id: I64d7ed92788ea0920d12cecd1aa0e414720e9b9c
This removes the indirect dependency to go.opentelemetry.io/otel, which
causes a dependency conflict in storj.io/ipfs-go-ds-storj.
Change-Id: I28c1e3637bb23f7949988511051d39c08b3fc852
This removes the indirect dependency to go.opentelemetry.io/otel, which
causes a dependency conflict in storj.io/ipfs-go-ds-storj.
Change-Id: I28c1e3637bb23f7949988511051d39c08b3fc852
This ensures that empty trash and restore trash cannot run at the same
time.
Fixes https://github.com/storj/storj/issues/5416
Change-Id: I9d2e3aa3d66e61e5c8a7427a95208bb96089792d
Adds new method Exists which can be used to verify which
requested piece ids exists on storage node. Will verify only pieces
which belongs to the satellite that used that endpoint.
Minum WASM size was increased a bit.
https://github.com/storj/storj/issues/5415
Change-Id: Ia5f9cadeb526541b2776a8973eb7d50133ad8636
* storj/common
* storj/private
Latests common version requires small refactoring for names and types
used by metainfo code.
Change-Id: I224fe93b4751c996ba6e846be0e5677252cf830f
Full file sync before saving files to piecestore seems to be very expensive.
Bwfore we do any step to eliminate them we are planning to do more measurement which requires a temporary flag to turn it off. (not for production).
Change-Id: I5cb8f8cb348ca3590fb5eae14d02edb3f0424617
Add backfill payments test to testsuite/storjscan.
Test ensures that even when something happens with
storjscan and it's unvailable from satellite perspective
that all payments made during storjscan downtime will be
picked up during next StorjscanChore cycle.
Change-Id: I4db699a29061023e159b3191a1e4dcfae6d0175e
Pieces count in DB are stored as int64 and we would like to align bloom
filter processing with this type.
Change-Id: Iaec767e609a40d802077ae057520541805a7c44f
Update storjscan API payment type to use currency.Amount for token and
usd values.
Update storjscan version for testsuite intgration tests.
Change-Id: Ie4ba7277b25d0d297a0130a8b885d6d38ff5eea4
Remove pkg satellite/payments/monetary as it moved to storj.io/common.
Update all code pkg references from monetary to common/currency.
Change-Id: If2519f4c80cf315a9299e6521a6b9bbc6c399156
As a reminder: latest clingy removed the requirement of having custom context (which made the usage of context.WithValue harder) and uses simple context instead.
Clingy saves the stdin/stdout/stderr to the context (earlier to separated context type) to make it available for unit testing.
Change-Id: I8896574f4670721de43a577cd4b35952e3b5d00e
Adds USDMicro currency which support fraction of a cent with decimal places
for better billing amounts accuracy.
Adds JSON marshaling and unmarshaling for monetary.Amount, so that it
can be converted to/from JSON.
Change-Id: I034eba120ed23b6ba00b2d81a4f1b9db5f9a203f
Adds test for storjcan chore which creates one tx for claimed
wallet, triggers chore to query new payments from storjscan and
uses service to retrieve payment.
Change-Id: Ic774213420f1b01d8adfeeb5ca8054d9c918e7b1
Create Run function which spins up storjscan, eth test network and
testplanet configured with storjscan endpoint and credentials.
Change-Id: I5bafd07b6b2e4f39228faf76af7dd7135bed5f77
We need to provide the ability to see bucket attribution on the gateway side
so customers can validate if bucket is attributed to them. Extendet metainfo.ListBuckets
request with UserAgent.
Fixes https://github.com/storj/storj/issues/4965
Change-Id: I5624874a7faa14cda06183ad44013e9ebb385b63
I would like to start moving some of the integration tests from scripts
folder into testsuite. This allows integration tests to easily have
other and fewer dependencies.
This will also be useful for https://github.com/storj/storj/issues/4823.
Change-Id: Idbdb5aefc0e42d091ad7f74dbdc096281c1dfcba
Implemented new modal to open bucket.
This is a part of new objects flow.
It is hidden by feature flag.
Change-Id: I3839126933b8804c8adbd8949a29ac75771fab4a
old bucket creation flow removed
new flow added
name and passphrase splitted into separate views
demo bucket will not be created automatically
bucket creation progress bar added
Change-Id: I2a1d7d77c3038caaafb3c06bdb0ac5dd1ad17599
We can use PieceIDDeriver in all places where we are deriving id from
the same id multiple times. We have serveral such places: gc, segment
deletion, segment validation, order limit creation. Using it should
save some resources.
Change-Id: I24668d516c0f7cea4aec6470614067734149501d
Added new modal when free tier user tries to create new project.
This modal prompts to upgrade account.
Change-Id: I0e787e35b099387a08186c7df7fc080092d2366c
We must always use 127.0.0.1 for testing and optionally support
customizing it for different scenarios.
Also few minor fixes to testsuite/ui/satellite.
Change-Id: I46ef151c6aa92768cb24fc2b8c768f10d5dea75d
To save load on DNS servers, the repair code first tries to dial the
last known good ip and port for a node, and then falls back to a DNS
lookup only if we fail to connect to the last known good ip and port.
However, it looks like we are seeing errors during the client stream
Close() call (probably due to quic-go code), and those are classified
the same as errors encountered during Dial. The repairer code sees this
error, assumes that we failed to contact the node, and retries- but
since we did actually succeed in connecting the first time around, this
results in submitting the same order limit (with the same serial number)
to the storage node, which (rightfully) rejects it.
So together with change I055c186d5fd4e79560f67763175bc3130b9bc7d2 in
storj/uplink, this should avoid the double submission and avoid dinging
nodes' suspension scores unfairly.
See https://github.com/storj/storj/issues/4687.
Also, moving the testsuite directory check up above check-monkit in the
Jenkins Lint task, so that a non-tidy testsuite/go.mod can be recognized
and handled before everything breaks weirdly and seemingly randomly
later on.
Change-Id: Icb2b05aaff921d0af6aba10e450ac7e0a7bb2655
Moved invalid email testing to separate test.
Made all the emails used to have .test domain.
Added links to regex resources.
Change-Id: I26920ba7360064528256a6aeaea947bbe56ef618
Uplink is fixed and now we should always get both key and nonce
or both empty.
Fixes https://github.com/storj/storj/issues/4646
Change-Id: I65dca2d4d5a10787c2fecad39e301121f1ae242a
Remote closing during upload or download is entirely expected and
it shouldn't lead to an error in the log.
Bump drpc to get the version that contains correct error code
for it. Also bump errs, which contains a fix for .Has.
Fixes https://github.com/storj/storj/issues/4609
Change-Id: I9297cabcfdc4b3a2c19d478dc729f779a2aef0c3
This change adds endpoints for supporting OpenID Connect (OIDC) and
OAuth requests. This allows application developers to easily
develop apps with Storj using common mechanisms for authentication
and authorization.
Change-Id: I2a76d48bd1241367aa2d1e3309f6f65d6d6ea4dc
Reworked email validation for new users (for old users trying to login or reset password validation remains the same).
Regular expression was built according to RFC 5322 and then extended to include international characters.
Change-Id: Id0224fee21a1ec0f8a2dcca5b8431197dee6b9d3
Copy object functionality should support setting new metadata for
copy. This change is adjusting FinishCopyObject method to set new
metadata when OverrideMetadata field is set to true.
Fixes https://github.com/storj/storj/issues/4483
Change-Id: Ica37cb57e8edae301cdc483fbda4f3ddba5d2702
In addition to upgrading the storj.io/common library, this change
moves off the TCPConnector in favor of the HybridConnector per
the deprecation warning.
Change-Id: I7e7e1e7568e8b95e4a99ad9caa158a799e68e1e3
Users signing up through a url containing a promo code will have that code applied to their stripe account instead of the free tier coupon.
Change-Id: I071041b0934648ef3f5bdb05b6ec97c400f89ae4
