Commit Graph

6059 Commits

Author SHA1 Message Date
Egon Elbre
51f488fc4b satellite/satellitedb: fix selection query with AOST
Union query for both reputable and new nodes didn't properly work. The
top level query is required to have an `AS OF SYSTEM TIME` statement as
well.

Change-Id: I8ee6dd5b700c2b1ed2aa562962bfa72be7eec30a
2021-10-19 16:59:40 +03:00
Michał Niewrzał
4a146000cc satellite/metainfo: read from DB only needed columns fro bucket
Uplink needs only part of columns we are reading from DB.
To improve performance we should read only those that are
realy needed.

Change-Id: Ib39259318169c46afe5fa4c6ce2184da82e960c8
2021-10-19 11:53:42 +00:00
Michał Niewrzał
8293c9659e satellite/metabase: speedup iterator tests
Main speedup is from reducing number of cases to iterate
in 'boundaries' test case.

Change-Id: I5bbf1311790f3d0e553d59e6a73e1cf3fa601a55
2021-10-19 11:52:07 +00:00
Michał Niewrzał
f5f2398431 satellite/metabase: remove code related to part deletion
It is not used currently.

Change-Id: I32e9fa357814a3159ee8659a607cd6986f361fbf
2021-10-19 10:11:49 +00:00
Moby von Briesen
4ad7056bf7 satellite/payments: Add old invoice list functionality
3b751a35c Removed our old coupon functionality, and slightly reworked
the invoice List() function in the stripecoinpayments package.

It turns out, this is causing some issues when trying to delete users.
This change keeps the new functionality, which is used in the satellite
UI, but under a new name, ListWithDiscounts()

Change-Id: I6a62a1de480e09d005dd22d75aa1e024fd2ed3a0
2021-10-18 18:18:18 -04:00
Yingrong Zhao
35e4a87e60 satellite/repair: ignore expired segments at the beginning of the repair
work

Since we have changed the repair worker to also mark a node as audit
failure if they return a not found error, we should ignore expired
segments when possible

Change-Id: Ie6a677e1d7b234e93965c736d05950440236653c
2021-10-18 18:15:39 +00:00
Clement Sam
29599dd7cd testsuite/ui/multinode: add first node test
Change-Id: Ibeaee92e6d1a41e408659a5045f52e1908f73089
2021-10-18 16:07:01 +00:00
dlamarmorgan
4bbf667ad1 satellite/{satellitedb,attribution,console}: value attribution changes that add userAgent field to buckets table and all tables that have partner_id
Change-Id: I36a13bb651b86bfc14fe5a0a2258f719e6cd2b48
2021-10-18 13:56:19 +00:00
nerdatwork
8a0a233537
web/satellite: Fix typo in 2FA popup (#4221) 2021-10-18 15:16:55 +02:00
littleskunk
016ac75052
satellite/console: Enable new onboarding workflow (#4222) 2021-10-15 21:07:35 +02:00
Malcolm Bouzi
d7812a3b17 satellite/console,web/satellite: add slider and unit toggling to project limit updating
Change-Id: Iab7c08baa09d598ff53ddf09ef09dcd2a997a81b
2021-10-15 14:11:50 +00:00
Egon Elbre
52c950e429 testsuite/ui/uitest: add Edge testing
Change-Id: I1e592ac755b582d4fa73e95fd999a35a27f8a116
2021-10-15 13:27:13 +00:00
Michał Niewrzał
96dc43fd49 satellite/metainfo: disable DeletePart method
We can now overwrite segments in pending objects so
we should stop using this method.

Change-Id: Ib24bd53e1a73d8c380b89a994a3cd8d178f4c533
2021-10-15 12:20:50 +02:00
Cameron Ayer
56fe636123 satellite/{reputation/satellitedb}: remove references to contained column in reputations table
We don't use this column for anything. If you want to know if a node is
contained, you can check the pending_audits table.

Change-Id: I5671722a5fc6e1749d3a49e187a56556000ff941
2021-10-14 19:59:03 +00:00
Cameron Ayer
bb21551a9c satellite/satellitedb: remove references to contained column in nodes table
We don't use this column for anything. If you want to know if a node is
contained, you can check the pending_audits table.

Change-Id: I8da1d8e01a2dcaff63c5067a7927b5451424ad04
2021-10-14 19:17:46 +00:00
dlamarmorgan
ac543714e0 add config file for git review usage
Change-Id: I7079a3a534f50b65c586c33d3d05ce6aa5aeb02f
2021-10-14 18:01:30 +00:00
Moby von Briesen
331132be95 web/satellite: Fix support link on registration page
The old one was pointing to storagenode support.

Change-Id: I0d16f0a8a21f70d3b5a49b514fb12e1836d3834a
2021-10-14 17:21:26 +00:00
Egon Elbre
8f4550d621 ci: fix using npm run build-wasm
Newer npm seems to invoke the command without the current user
privileges. However, Go already initializes its cache using root.

Change-Id: I23b32ca3bbd7db06515c2af86b1bb5adc8e38a8f
2021-10-14 15:10:40 +00:00
Egon Elbre
c54bcd63a1 Makefile: bump node to v16
Change-Id: I0587168f373230710c7b50b9aefd34718f4a5b3e
2021-10-14 12:46:28 +00:00
Khushal Shah
4535da96ee
web/multinode: removed conversion for Expectations (#4216)
What: 
Removed the `convertToCents` methods from the `Expectation` class, since the expectations were already in cents.

Why:
Fixes https://github.com/storj/storj/issues/4202
The `/expectations` endpoint already returns the amounts for `currentMonthEstimation` and `undistributed` in cents. So there is no need to convert them again to cents before the dollar conversion.

This change removes the `convertToCents` from the `Expectation` class.
2021-10-14 12:12:24 +00:00
Jeff Wendling
91ce70e4ed satellite/metabase: allow overwriting segments in pending objects
this is so that multipart upload allows overwriting parts. our
consistency model is more relaxed because part overwrites
won't be atomic if they consist of multiple segments, but at
least this allows overwriting at all.

Change-Id: I21dac4c24046e05efe74e6c6fd189a02c95eb6c8
2021-10-14 10:33:52 +02:00
Moby von Briesen
5b729779a2 satellite/console: Automatically log a user in after verifying email
When an email is verified, insert an auth cookie so that when the user
is redirected after verifying their email, they are immediately taken to
the onboarding flow.

Change-Id: I557d8a2805b24dd8039ada255522bc1b56cc8b53
2021-10-13 13:08:27 +00:00
Clement Sam
b8dd35ceaf private/testplanet: extend testplanet with multinode instance
Testplanet does not have a multinode instance, hence,
makes it difficult to run ui tests for the multinode dashboard.

Instead of using storj-sim for multinode ui tests,
it's better to use the same approach (testplanet) fir all UI tests.

This changes adds a multinode instance to testplanet.

Change-Id: I58aa8c4597e789275f9e7ea7059703c742903492
2021-10-13 10:19:45 +00:00
Michał Niewrzał
b8e8110ca3 mod: bump common and uplink dependency
Contains also adjustments for latest storj/common changes.

Change-Id: I11f70d466b3d9785ee93ac424629b2b868889d02
2021-10-12 19:13:49 +00:00
Egon Elbre
aa858a3a23 testsuite/ui/uitest: separate func for browser
The code will be clearer if the browser starting is entirely separate
from the testplanet and edge setup.

Change-Id: I6d26aa25f4c2a929ba4eda1388234e8aeec518c7
2021-10-12 18:09:48 +00:00
Michał Niewrzał
1fdb0eaa5b Revert "satellite/metabase: use storj.Nonce instead []byte"
This change introduce problems with server side move so
let's revert it for now. Problem was found when latest
version of storj/storj was used in uplink tests.

This reverts commit 1ef06fae99.

Change-Id: I4d4fad5d1ea04ba15ff9d7bd765f7e078e9187c2
2021-10-12 15:39:54 +02:00
Egon Elbre
bc6d8c06e3 web: optimize builds
Stylelint and eslint should be run separately and not part of the build
process.

Add a flag STORJ_DEBUG_BUNDLE_SIZE to debug compiled bundle size.

Reduce the number of chunks, it's far from ideal. Once we reduce the
images and browser size, we probably can drop chunking altogether.

Change-Id: I5bdf35ceb140e2c47a30df8d319606d05bfb30dd
2021-10-12 11:29:37 +00:00
Egon Elbre
4020e9e2ce private/testmonkit: move monkit test helper
Some tests do not use testplanet and hence testmonkit should be
reusable.

Change-Id: If6b7194a199d9ffba5eb1a91f38d3a792f883336
2021-10-12 14:00:01 +03:00
igor gaidaienko
5b4a9070b0 scripts/testversions: reduced the number of version to test 2021-10-12 13:28:46 +03:00
Jeremy Wharton
3b751a35c5 satellite/{payments,satellitedb}: Remove custom coupon implementation
Removes database tables and functionality related to our custom
coupon implementation because it has been superseded by the Stripe
coupon and promo code system. Requires implementations of the
payments Invoices interface to return coupon usages along with
invoices.

Change-Id: Iac52d2ff64afca8cc4dbb2d1f20e6ad4b39ddfde
2021-10-11 19:47:00 +00:00
Michał Niewrzał
1ef06fae99 satellite/metabase: use storj.Nonce instead []byte
We were using mixed types for nonce fields. Protobuf
have storj.Nonce, metabase have []byte. This change
is a refactoring to have everywere its possible only
storj.Nonce.

Change-Id: Id54bd8481f30c721cdaf3df79206d25e7cfdab55
2021-10-11 16:13:34 +00:00
Egon Elbre
b7405db2a2 satellite/satellitedb: optimize migration test
Cockroach doesn't like concurrent database creation, add a limiter to
avoid overloading the database.

Also limit the cockroach testing to the last 10 migrations.

Change-Id: Ie73c516ac2d362b251f605049e51eb25888432bd
2021-10-11 18:31:20 +03:00
Fadila Khadar
f77f61532a satellite/orders: use egress_dead for calculating allocated bandwidth
Change-Id: I8240d99d0cdaad4c5e059565e88ee9619b62526e
2021-10-11 14:58:26 +02:00
Egon Elbre
9fd091831d ci: optimize benchmarks
We are not using the benchmark results for anything, they are mostly
there to ensure that we don't break the benchmarks. So we can disable
CockroachDB for them.

Similarly add short versions of other tests.

Also try to precompile test/benchmark code.

Change-Id: I60b501789f70c289af68c37a052778dc75ae2b69
2021-10-08 19:42:40 +03:00
Michał Niewrzał
f837551d76 storj: bump uplink to v1.7.0
Change-Id: Ic5ffc704ffbcf7dc890fe0554a8e23f1a777ecbd
2021-10-08 08:04:57 +00:00
Vitalii Shpital
1d8342ab17 web/satellite: fixed access grant web worker caching issue
Fixed caching issue by using regular Webassembly.instantiate instead of instantizteStreaming which fails to work in chrome sometimes.
Now web worker instantiation is part of initial app loading process. This fixes go-rod test flakiness.

Change-Id: I5300e70d4e0866f2321df86ce729f32b588bc5c4
2021-10-07 14:31:04 +00:00
Ivan Fraixedes
05960b2cf0
satellite/admin: Response 404 when entity not found
* Add test cases to verify that all the endpoint that target a specific
  entity respond 404 status code when the entity isn't found.
* Fix the endpoints that target a specific entity which responded a 500
  status code response when the entity didn't exist to respond with 404
  status code.

Additionally:

* Simplify some tests using an existing test helper function.
* Rename test functions to start with the entity name (e.g. Project,
  User, etc.) for easing to run a set of test with the `-run` Go test
  flag.

Change-Id: I82aad92e429207b72932ad4b79c08db6b486a19a
2021-10-07 12:42:25 +02:00
Fadila Khadar
fb0d055a41 satellites/orders: populate egress_dead in project_bandwidth_daily_rollups
Populate the egress_dead column for taking into account allocated bandwidth that can be removed because orders have been sent by the storage nodes. The bandwidth not used in these orders can be allocated again.

Change-Id: I78c333a03945cd7330aec052edd3562ec671118e
2021-10-06 16:54:49 +00:00
Egon Elbre
6ab2647adb ci: run UI tests
We have implemented few UI tests using go-rod, but haven't been running
them regularly.

Change-Id: I6c89365f719882431842a2026b426860f1293b8b
2021-10-06 15:54:06 +00:00
Michał Niewrzał
8a21a8cbc5 satellite/metainfo: don't enable connection pool if was setup earlier
It's safer to create new connection pool for piece deletion
only if dialer have no existing pool assigned.

Change-Id: I26661683ab7c0198587905478057c01c8f533a7e
2021-10-06 15:25:23 +00:00
Yingrong Zhao
5355928317 satellite/payments/stripecoinpayments: fix nil panic during storj token
deposit

Change-Id: I2294656aad475748b53d786b03c3810b338963bb
2021-10-05 12:13:14 -04:00
Michał Niewrzał
e5bb897364 satellite/internalpb: for StreamID rename EncryptedPath to EncryptedObjectKey
We should be using object naming insted of path.
This is one place where we can easiliy change it.

To regenerate protbuf I had to remove gogo.proto.
Most probably it was confilicting with gogo.proto
from common/pb.

Change-Id: Ia5972f77994765c8f26bf1c3dc8205d2eadd70fa
2021-10-05 15:15:49 +00:00
Michał Niewrzał
f93dc5a166 satellite/metainfo/piecedeletion: enable connection pool
We want to enable connection pool for piece deletion to avoid
doing multiple SSL hanshakes to SN while massive deletion process.

Change-Id: Ic917e4eda304ee16a286926ef046fe9e38bf38ca
2021-10-05 13:30:42 +00:00
Egon Elbre
bb55c30594 web: move eslint-storj to separate repo
For some reason our build process cannot handle relative path packages,
it's easier to create a new repository than to figure out,
why npm/docker etc. don't like it.

Change-Id: I94b7cb9611f453246b596f97114fe5c54d9a4008
2021-10-05 12:08:44 +00:00
Michał Niewrzał
aa80d37053 satellite/metabase: add automatic conversion to DB value for ObjectKey fields
We were manually converting ObjectKey fields to []byte to use it with
SQL query but we can just implement Value method to convert it automatically.

Change-Id: I6d346f4b59718e1e8ef37cd9f95e613b864b42cd
2021-10-05 07:34:00 +00:00
Yaroslav Vorobiov
4b79f5ea86 satellite/repair: test if audit scores increases during repair
Update repair tests to check if audit score increases for nodes
that successfully send pieces during successfull and failed repairs.

Change-Id: Ie6abbde6155ab4697d209366c9fa497e731756e9
2021-10-04 19:39:13 +00:00
Fadila Khadar
38ce0e154c satellite/gracefulexit: drop table graceful_exit_transfer_queue
Drop table graceful_exit_transfer_queue which is not used anymore (replaced by graceful_exit_segment_transfer_queue).

Change-Id: Ie254fe9a54fb0784e350a439ce7a9bc99a3a58b5
2021-10-04 17:26:44 +00:00
Michał Niewrzał
f2f8a9ca1e satellite/metabase: send metric about zombie segment deletion
We would like to verify if zombie object/segment works fine.
We need some metric for that. Figuring out number of deleted
objects is harder so let's for that later.

Change-Id: Ic99e2ce93256130b7c51f514824fddc009655075
2021-10-04 16:25:23 +00:00
paul cannon
5f444c1fee satellite/payments: fix currency rates acquisition
Change-Id: I4667b9d0131efe04f5429e932cbb0f0da8699409
2021-10-04 15:27:34 +00:00
Malcolm Bouzi
10a468fb50 web/satellite: make all header components visible on smaller screen sizes
Change-Id: If84ad344a40bb666250106d2f2b9ffaa01413296
2021-10-04 13:55:31 +00:00