Currently the rate limit has kept per satellite api endpoint.
Since we run 9+ api endpoints in production, we do not need
a limit of 1000, since the intention was to allow 1000 total.
This change reduces the effective limit given 9 instances
down to 900, which should be close enough.
Change-Id: Ia579149ccc3a12e8febe0cfd5586b8a39de40f55
Free-tier segment usage limit was defined as 150k, not 140k. This change
is correcting that.
https://github.com/storj/team-metainfo/issues/8
Change-Id: I71ec0961930b19fd09b2b996e01acd406a8dcf8f
We want to be able to limit the number of segments per project for users.
To limit this we need to check limit value associated with project
and value of used segments already in BeginMoveObject, BeginMoveSegment
and increment cache segments usage after each CommitSegment call.
Resolves https://github.com/storj/team-metainfo/issues/1
Change-Id: I6290e67c095a174b9d101c4521802d9bfe0453b8
this makes the flags match rclone nomenclature
fixes test-uplinkng to use the temporary config dir
instead of the machine default, and clean up some.
bumps clingy so that the command errors when an unknown
command is specified.
also fixes some printfs in share to use clingy stdout.
it still does some external actions that should be
passed through a ulext.External for mocking, but
that's ok for now.
Change-Id: Icc231e7e26393541c312396fec907b640b97718e
This change adds a script and the needed build logic to create a draft github release and then upload the created binaries to it in order to make the actual publishing a lot less error prone.
The logic is currently that it only does this for all github tags, but not for every main build/push. This is handled by the checks in the script itself.
Change-Id: Ie172a8e4a97200de901a26a055aa5a8a54b60a2a
We enabled this chore on different satellites
for testing. Works fine so we can enable this
by default for all configurations.
Change-Id: I987639685d8de5c7e5798adca30fe26bdac9e1d1
All limits we have for projects have also parent limits stored
with user data. New created project is first taking limits from
owner (user) limits.
This change is extending users table with project_segment_limit
column and adds functionality to get and set value for this
column.
Change-Id: Iff5e36c62b517652390b649fc05992475916ecff
This change disallows creation of users possessing the same email.
If a user attempts to create an account with an email address
that's already used - whether it belongs to an active account or not -
he will be notified of unsuccessful account creation. If he attempts to
log in using an email address belonging to an inactive account,
he will be presented with a link allowing him to re-send the
verification email. Attempting to register with an email address
belonging to an existing account triggers a password reset email.
Change-Id: Iefd8c3bef00ecb1dd9e8504594607aa0dca7d82e
this allows commands like
uplinkng cp -r sj://foo sj://bar
to work correctly, rather than complain that sj://foo is
not a boolean.
Change-Id: I003e47aabb85566bc2b454851cf55043b17ee7ea
To allow for changing limits for new users, while leaving existing users limits as they are, we must store the project limits for each user. We currently store the limit for the number of projects a user can create in the user DB table. This change would also store the project bandwidth and storage limits in the same table.
Change-Id: If8d79b39de020b969f3445ef2fcc370e51d706c6
Change the satellite Admin HTTP server for:
* Embedding the UI assets into the Go binary.
* Serve the UI assets from the embedded file system or from a specific
directory path through a configuration flag, without requiring
authentication but keeping the authentication verification for the API
endpoints.
* Add tests to verify that the UI assets are served without
authentication.
Change-Id: I9003ac96f1ec585a189b67fc1cb315905403d557
This change adds the ability to download byte ranges
to uplinkng.
Extended the uplinkng Filesystem interface with Stat
method and an OpenOptions struct as parameter for the
Open method.
Also added a few tests for the ranged download
Change-Id: I89a7276a75c51a4b22d7a450f15b3eb18ba838d4
Gateway-ST frequent release cycle has been resurrected, which means it's
safer to use the latest release tag in the storj repository's CI now.
Change-Id: I9df1c789a9b9418ba7cceaec9cfec3cc6c448284
Currently slower storagenodes can slow down deletion queue.
To make piece deletion faster reduce the maximum time spent in
either dialing or piece deletion requests.
With this change:
* dial timeout is 3s
* request timeout is 15s
* fail threshold is set to 10min
Similarly, we'll mark storage node as failed when the timeout occurs.
The timeout usually indicates that the storagenode is overwhelmed.
Garbage collection will ensure that the pieces get deleted eventually.
Change-Id: Iec5de699f5917905f5807140e2c3252088c6399b
To make our free tier limits more clear, we will reduce the number of projects allowed from 3 to 1, and increase the storage and bandwidth limit of the free tier from 50 Gb to 150 GB. The total allotments across all projects for a given user are unchanged, just reduced to a single project.
Change-Id: Ic8dddb135f2b83a3f36e2b9fdcb477e351ec137b
Multipart upload limits added. Last part has no size limit.
Max number of parts: 10000, min part size: 5 MiB
Change-Id: Ic2262ce25f989b34d92f662bde720d4c4d0dc93d
https://storjlabs.atlassian.net/browse/PG-305
we should extend method move of cmd/uplink/cmd/mv.go
if both parameters end with slash -
i should list all files and call move method in loop
parameters:
uplink mv sj://bucket/a/prefix/ sj://new-bucket/a/new-prefix/
Change-Id: Ic24c2af83153ea60ec74393e65736af094877151
Removes database tables and functionality related to our custom
coupon implementation because it has been superseded by the Stripe
coupon and promo code system. Requires implementations of the
payments Invoices interface to return coupon usages along with
invoices.
Change-Id: Iac52d2ff64afca8cc4dbb2d1f20e6ad4b39ddfde
New command for cli to move object to different
location.
uplink mv sj://bucket/your-object sj://bucket/moved-object
Change-Id: I85a4961aa59f250819954e78f20363ac3c570938
Make bucket command was using full location
specified in command line instead only bucket name.
As an addition change contains basic integration tests
with storj-sim.
Change-Id: Ie3b5283468b7fbde0b1333f01dc4fc2a2952e1a1
Jenkins uses the same folder for different PR-s. Sometimes other runs do
not cleanup after themselves (e.g. timeout), hence add a cleanWs step to
ensure we delete files in the workspace.
gateway-st introduced a replace directive in go.mod, which does not work
with go install. Hardcode to the last version without the directive.
Using this fix to unblock ci builds.
Change-Id: I5e5d75bf47e30a5a8b6d835867c0c9176f25e08a
Use multipart upload to upload single object in parts in
parallel. Its using parallelism flag added earlier.
Change-Id: I45b531a5db43c86f0112a5e3bb4a83bc1d65650f
Adds support for new uplink method DownloadObjectAt which
gives ability to download single object in parallel.
Change-Id: I8388653429992b0d24c383d17d7e90904203fe77
Rate limits application of coupon codes by user ID to prevent
brute forcing. Refactors the rate limiter to allow limiting based
on arbitrary criteria and not just by IP.
Change-Id: I99d6749bd5b5e47d7e1aeb0314e363a8e7259dba
MFA is complete and we are good to enable it in production. This change
removes the flag that disables MFA by default.
Change-Id: I2f985ae501171bdab505d664b43c8cfc248bad8d
