The ApplyUpdates() method on the reputation.DB interface acts like the
similar Update() method, but can allow for applying the changes from
multiple audit events, instead of only one.
This will be necessary for the reputation write cache, which will batch
up changes to each node's reputation in order to flush them
periodically.
Refs: https://github.com/storj/storj/issues/4601
Change-Id: I44cc47767ea2d9423166bb8fed080c8a11182041
Implemented new modal to open bucket.
This is a part of new objects flow.
It is hidden by feature flag.
Change-Id: I3839126933b8804c8adbd8949a29ac75771fab4a
Implemented project delete endpoint for REST API.
Added project usage status check service method to indicate if project can be deleted.
Updated project invoice status check method to indicate if project can be deleted.
Change-Id: I57dc96efb072517144252001ab5405446c9cdeb4
prevent network enumeration by rejecting privateIPs in PingMe and
Checkin endpoints
Closesstorj/storj-private#32
Change-Id: I63f00483ff4128ebd5fa9b7b8da826a5706748c9
We don't have metric to track how many pending objects we have in the
system. This change is using tally objects loop to collect pending
objects per bucket and at the end its combining all buckets values
into single metric for all pending objects in a system.
Change-Id: Iac7a6bfb48854f7e70127d275ea8fdd60c4eb8b7
Created and style modal-added open and close functionality
Added logic to access grants selection type
Changed CLI to API-Changed Types to radio buttons-Added date picker-Added buckets list(not yet populating with buckets)
Added permissions check box logic and new input field for buckets.
Added tooltips and hover logic for the tooltips
Added acknoledgement functionality to encrypt step
Added conditional logic to encrypt access button and the ascknoledgement box - removed blank comments from paymentMethods snapshot
Added logic to retrieve restricted api key, access credentials, and satellite address
Added copy functionality and download functionality for the credentials step
Change-Id: I8c8f02bc1ee38c3df42396cbd9bb3db2e7ff9cc4
Co-authored-by: cl-mitch <mitch.george@compozelabs.com>
In the name step of the create bucket view for the new objects flow, fix
the placeholder for the input to mention "name" rather than "passphrase"
Change-Id: Ia7a64383b58d36f6aee1df952ed7cb300734e09d
Add storjscan wallets implementation to the satellite. The wallets interface allows you to add and claim new wallets as called by the API. The storjscan specific implementation of this interface uses a wallets DB to associate the user to a wallet address, as well as a storjscan client to request and associate new wallets to the satellite.
Change-Id: I54081edb5545d4e3ee07cf1cce3d3e87cc00c4a1
Update all the NPM dependencies used by the Admin UI.
The dev dependencies correspond to the ones that are currently used by
an svelte app generated with the last svelte-kit version. They
deprecated some configuration options and changed some svelte
directives.
The only non-dev dependency is also updated to the last published.
Change-Id: I5f2192cab41e00efc3239237f8dc8f3d07816b63
script-src-elem is preferred over script-src in certain scenarios.
If it's absent, then the browser always uses script-src. By adding
script-src-elem it ended up blocking google recaptcha.
Change-Id: I9cf96e71e69054c4a034ca189db84fbe8903a59b
This change adds project ID and bucket name columns to the generated
partner attribution report. Attribution values are now summed based on
their project ID and bucket name in addition to their user agent.
Additionally, the command to generate the attribution report has been
modified to optionally include only certain user agents.
Change-Id: I61a1d854379134f26b31467d9e83a787beb451dd
Updated daily project usage query to return correct allocated traffic.
If allocated egress has expired then we return settled egress.
If not then we return allocated egress - dead egress.
Fix for this issue
https://github.com/storj/storj/issues/4563
Change-Id: Ia15a50d3bb8d8cb1106936e17dbe0f1f5a40fa87
Fixed daily usage query returning single bucket usage.
We sum up bucket usages now.
Also fixed https://github.com/storj/storj/issues/4559.
Change-Id: I2eb6299f1ef500d68150879195011b6fbb5f37ed
The admin UI assets aren't inside of the `web` directory they are
directly in the `satellite` one.
The invalid path provoked that storj-sim generated a satellite
configuration with an invalid path to the Admin UI static assets
provoking that it didn't load the UI by default.
Change-Id: I49fb289377f51634057173690fbd8cf863ca9a9d
TRUNCATE requires table recreation which involves 'online schema change' with crdb.
(with psql it might be fater than DROP, that was the motivation of the original change)
`online schema change` is an async operation with crdb and it's eventually very slow therefore we try to avoid it.
This·reverts·commit·15bed0ed0e81d54fe4ffac9928bdf648f5e06ec6.¬
Change-Id: I93e1ab470962be77e3458d74c8787442c9d7bee0
This patch suggests small improvements for the previously committed (very nice <3) unit test helper
* Use non-default ports for postgres + cockroach. This would help to run same services for ad-hoc unit test (for IDE development for example)
* Use the same flags for cockroach what we use for unit tests (mem store supposed to make tests faster)
* Use `docker-compose down -v` for removing all networks and annynmous volumes
* Remove connection limit (I have seen related problems in my jenkins runs)
* Omit unused databases to avoid getting failures (eg. omit psql when crdb is used)
* Do not drop cockroach database (full database will be dropped, anyway...): it also makes testing faster
Change-Id: Iadc04f8617a2825ea4f10072bd023a9c86883680
Main issue was that when one part copy failed while being inside
goroutine (limiter) and another part was still collecting src/dst parts
it was possible to drop errors from failed part copy. It was possible
bacause on fail context was canceled and if we were still getting
part src/dst then it was returning error immediately and error
group with errors from goroutine was ignored.
Change-Id: I75c6799eba358741629795f2971c7a964cb2c9ce
Fixed main content width to be appropriate.
In case of very long access grant name (100+ symbols) main content gets overlapped with window border. That's an unhandled edge case.
Change-Id: I42ef709edc8454f054baa52231c605afadc471e2
Add the admin API endpoint for disabling an user's multifacdtor
authentication to the satellite admin UI.
Remove a couple of commented code lines too.
Change-Id: Iaee7efe7a3d4d38bdd6541311447a9726806f0f1
We have a couple of support tickets so far that require us to
disable the mfa on accounts. Since we currently had no other
way than doing a SQL War Crime, it makes sense to add it to the
admin API.
Change-Id: Ib16735c1961380b04345a3495d4eebee5fa0bc41
Currently we have a bug in which we would require that a project of
a paid tier user needs to be two months unused before we can delete it.
This change fixes it and reduces it back to the normal next billing cycle.
Change-Id: I28610b6c45c68943fd4f2621233bccc06cab28a0
An older change plummed the full console config as subconfig of
the admin api configuration in. This bloated the generated satellite
configuration unnecessarily while also allow for confusion/mistakes.
Change-Id: Icf49cc1f147711e37e85f6eac1143fab8ddf1659
when deleting an object that has been copied multiple times, we look for an ancestor_stream_id by taking the min of all copies stream_id.
This change simplifies this process by picking any stream_id as a new ancestor by using 'distinct on'.
Fixes https://github.com/storj/storj/issues/4745
Change-Id: Iffb519b82d2ae2ed73af48fa0e86f87384e0158f
Few improvements were made to how we are handling errors
while doing parallel upload/download for single object:
* unhide error under 'context canceled' which was shown in most of
cases
* add part number to error message
* don't try to commit if any error occurs while operation
* combine errors into more readable form, example:
---
failed to download part 3: uplink: eestream: failed to download stripe 0:
error retrieving piece 00: ecclient: piecestore: rpc: tcp connector failed: rpc: dial tcp 97.119.158.36:28967: i/o timeout
...
error retrieving piece 89: ecclient: piecestore: rpc: tcp connector failed: rpc: dial tcp 161.129.152.194:28967: i/o timeout
failed to download part 1: uplink: eestream: failed to download stripe 0:
error retrieving piece 01: io: read/write on closed pipe
...
error retrieving piece 97: io: read/write on closed pipe
failed to download part 2: uplink: eestream: failed to download stripe 0:
error retrieving piece 00: io: read/write on closed pipe
...
error retrieving piece 01: ecclient: piecestore: rpc: tcp connector failed: rpc: dial tcp 180.183.132.234:28967: operation was canceled
error retrieving piece 96: io: read/write on closed pipe
main.(*cmdCp).parallelCopy:418
main.(*cmdCp).copyFile:262
main.(*cmdCp).Execute:156
main.(*external).Wrap:123
github.com/zeebo/clingy.(*Environment).dispatchDesc:126
github.com/zeebo/clingy.(*Environment).dispatch:53
github.com/zeebo/clingy.Environment.Run:34
main.main:26
runtime.main:250
---
Change-Id: I9bb70b3f754567761fa8d17bef8ef59b0709e33b
Reworked "add payment method" and "create project prompt" modals to use new common VModal component.
Change-Id: I73039e5d30077e35f05501a48f39142b799da875
`os/exec.Cmd.CombineOutput` runs the command, hence, it cannot be used
after calling the `Run` method.
Because `CombineOutput` already runs the command, we can use it directly
instead of `Run`.
Without this change if the command returns an non-zero code we get an
error because of the command already started and we don't get the
output.
Example removing a copyright notice from one file and running the linter
(only showing the affected printed line)
Without this fix
2022/05/26 15:48:40 [/storj check-copyright] error exec: already started
With this fix
2022/05/26 16:22:40 [/storj check-copyright] error missing copyright certificate/doc.go: %!w(<nil>)
NOTE the `%!w(<nil)` is a bug in the check-copyright linter.
Change-Id: I40b64842028399b92a8982bfb143e1f87f92467b
Add billing DB to the satellite. This DB will hold all transactions on the users account and can be used to compute the users current usable account balance.
Change-Id: I056416efc169e5e5e30c9f30cd8bc766b7bc8073
Implemented new service method for generating API keys.
Implemented new endpoint.
Improved multiple endpoint groups handling.
Change-Id: Iba26fbf9123707b5b4c2d5e8c5a35d507404f24a
We are not using this method and most probably we
won't need to list objects with all statuses at once.
Removing for now.
Change-Id: I7aa0468c5f635ee2fb1fe51db382595c6343dd9c
- parallel deletion of 50 objects and their 50 copies (one copy per object)
This test is skipped because it's creating deadlocks that are not automatically retried on postgres
- parallel deletion of 1 object and its 50 copies.
Fixes https://github.com/storj/storj/issues/4745
Change-Id: Id7a28251c06bb12b5edcc88721f60bf7a4bc0492
Our linting process depends heavily on custom tools and linting
configuration. To help improve this process for running on local
developer machines, we can run the various tasks in our existing CI
container.
Change-Id: I60407686ce9233cc4f16e3724c5e8d44367aa200
This change attempts to run tests as close to CI as possible. It
introduces a docker-compose file that deploys some supporting
services. While this change makes it possible to run tests locally,
it does not address the core time it takes to run tests end to end.
make test/postgres 1417.87s user 263.38s system 291% cpu 9:37.73 total
Resolves https://github.com/storj/dev-enablement/issues/10
Change-Id: I4d9adc125992c1b1c133e8fe108384fa3c66ff7a
testplanet executes cockroach and postgress tests parallel, therefore using http.DefaultClient is safe only as long as we don't modify it.
TestActivationRouting modifies it (client.CheckRedirect=...), therefore it should use a local version instead of the default one.
Problem reported by a jenkins build:
```
==================
WARNING: DATA RACE
Write at 0x000003486af0 by goroutine 143:
storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1()
/home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:66 +0x378
storj.io/storj/private/testplanet.Run.func1.1()
...
Previous read at 0x000003486af0 by goroutine 104:
net/http.(*Client).checkRedirect()
/usr/local/go/src/net/http/client.go:494 +0xd73
net/http.(*Client).do()
/usr/local/go/src/net/http/client.go:691 +0xd31
net/http.(*Client).Do()
/usr/local/go/src/net/http/client.go:593 +0x204
storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1.1()
/home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:48 +0x1e5
storj.io/storj/satellite/console/consoleweb_test.TestActivationRouting.func1()
/home/jenkins/workspace/storj-testing-experiments/satellite/console/consoleweb/server_test.go:74 +0x49d
storj.io/storj/private/testplanet.Run.func1.1()
...
```
Change-Id: I73319a5a593e067b906ec1fda70a44ca1e5a49a2
This has been a cause of some confusion, even though the fields are
labeled as being copies of config values.
Having them be under a field explicitly named "Config" makes this
clearer, plus, allows the values to be passed in simply as a copy
of the Config struct from the satellite, rather than copying the fields
individually (which can be error-prone, particularly as the AuditCount
field in UpdateRequest is apparently not the same thing as the
AuditCount field in reputation.Config).
Refs: https://github.com/storj/storj/issues/4601
Change-Id: I386953347b71068596618616934aa28e3245cdc1
