JakeHillion/storj
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4,501 Commits 5 Branches 0 Tags 94 MiB
4cbd4d52a9
Commit Graph

4501 Commits

This Branch
This Branch
All Branches
Author SHA1 Message Date
Egon Elbre
a4c554f2ed satellite/admin: support user query by email 
This adds new endpoint /api/user/{user-email} which allows to get the
projects where the user is a member.

It also moves existing endpoint:

    /project/{projectid}/limit -> /api/project/{projectid}/limit

To avoid future conflicts for displaying pages.

Change-Id: I5efe3e1c8f79894c136f92ed815f635a34ba6f98
 2020-04-06 18:32:25 +00:00
Qweder93
0f71e60a53 storanode/version/chore notifications temporary disabled 
Change-Id: Ifacdede772bc7b678288410d215e665b014f566f
 2020-04-06 16:57:12 +00:00
VitaliiShpital
61b6ff5186 web/satellite: available credits amount added on billing page 
Change-Id: I9d9c18ad08cf150ceba078356e39132a602558d4
 2020-04-06 14:39:05 +00:00
Michal Niewrzal
2d1a6968b0 satellite/metainfo: revert returning segment size and inline segment 
size with BeginObject

Such solution will add one round trip to satellite during upload so for
now we are reverting this until we will have solution for this.

Change-Id: Ic2d826448ab7b0318cd6922df05deee9167cf2f0
 2020-04-06 13:36:34 +02:00
Jennifer Johnson
1547e791a3 satellitedb: remove free_bandwidth column from nodes table 
Change-Id: I9d1d3de9216c6533c1042ef473631721a011d086
 2020-04-06 09:30:28 +00:00
Michal Niewrzal
b17dc656b5 lib/uplink-gomobile: update to latest master 
Change-Id: I1be7991ee73cc2a6e911ae024dce3fba97c02071
 2020-04-04 19:08:48 +00:00
Yingrong Zhao
4ab553cf3f scripts: only latest releases have version file 
Change-Id: Ice212ac39b011c18b1d3d48ea9e6f580b4fd7f0c
 2020-04-04 18:47:33 +00:00
Cameron Ayer
42be4bdc0f satellite/contact: add timeout to PingBack method 
Change-Id: I2ec2f82e2e10d8be16f82e9de13ce42358e47c98
 2020-04-04 18:26:30 +00:00
Egon Elbre
9200efc61f satellite/satellitedb: fix selecting a nullable string 
Change-Id: I59e645966e09da586512c69101691b47055c1e5a
 2020-04-03 21:30:20 +03:00
paul cannon
0c8c11b251 satellite/audit: add not_enough_shares_for_audit counter 
We have been using the SQL expression `name='(*Verifier).Verify' AND
error_name='not enough shares for successful audit'` thus far to detect
cases of this problem and alert on them. Unfortunately, since this
rarely (hopefully never) happens, influxdb has no data for most of the
auditor instances, and when it has no data for a time series, it returns
no columns either. This makes Redash upset when it tries to perform a
query for an alert and can't find the column whose value it expects to
check.

This change should make it so zero values are reported when the problem
has not happened, and higher values when it has.

Change-Id: I79e5e000f879678b661dac88caae1e2915b39ab1
 2020-04-03 17:00:50 +00:00
VitaliiShpital
dbd036de8b web/satellite: api key and proj member name's length visibility extended 
Change-Id: If42dc48849f6414f4224d57ffa7267431426b457
 2020-04-03 11:32:41 +00:00
Matt Robinson
4b01a8dd18
Descellate Usage Error to Debug (#3780) 2020-04-03 13:20:20 +02:00
sixcorners
06cddb053e
cmd/identity: improve CA key message (#3819) 2020-04-03 13:00:45 +02:00
Stefan Benten
0a4d253990
storagenode/storagenodedb: Improve preflight schema error message (#3844) 2020-04-03 11:20:24 +02:00
Yingrong Zhao
b3939bc1ba scripts/tests: fix test-version and rolling-upgrade test installation 
Change-Id: I2c1262aa2e4fd1f5aa34bffe9c76f272fe076d49
 2020-04-02 22:07:06 +00:00
sixcorners
4f8cf53c9e
Remove VOLUME /root/.local/share/storj/storagenode (#3820) 2020-04-03 00:05:20 +02:00
Jeff Wendling
a409bd5dec satellite/orders: check for expired orders first 
there are a subset of storagenodes hammering the satellite with
expired orders. if we check for expiration first, we don't have
to do a bunch of pointless signature verification. since a && b
is equal to b && a, we can order these checks in any way we want
and have it still be correct.

Change-Id: I6ffc8025c8b0d54949a1daf5f5ea1fed9e213372
 2020-04-02 12:35:11 -06:00
Egon Elbre
6492b13d81 all: remove old uuid 
Change-Id: I3a137f73456f010c37d3933dbe12cbbb840b809f
 2020-04-02 19:30:36 +03:00
Jess G
e725535873
cmd/satellite: add gc to entrypoint (#3842) 
Change-Id: I49816628bc4301006e4a54b5582b3d759806b5e7
 2020-04-02 08:39:13 -07:00
Egon Elbre
1024bf9ce1 all: simplify uuid usage 
Instead of uuid.Parse, use uuid.FromString.
This removes a bunch of pointer management logic.

Change-Id: Id25bd174eb43c71d00b450158a198abafd8958f2
 2020-04-02 13:45:19 +00:00
Michal Niewrzal
fe2340285f storagenode/console/consoleserver: fix TestConsole tests 
https://storjlabs.atlassian.net/browse/SG-145

Change-Id: Idee6050cea5fd53ec631a7c64fb18f1ad7f16af0
 2020-04-02 13:17:04 +00:00
Michal Niewrzal
c178a08cb8 satellite/metainfo: add max segment size and max inline size to 
BeginObject response

We want to control inline segment size and segment size on satellite
side. We need to return such information to uplink like with redundancy
scheme.

Change-Id: If04b0a45a2757a01c0cc046432c115f475e9323c
 2020-04-02 12:41:28 +00:00
Michal Niewrzal
4a79b609e9 satellite/metainfo: fix panic when we batch BeginObjectDelete without 
all permissions

Without read and list permissions BeginObjectDelete won't return error
if occurs. This was breaking Batch processing because there was
assumption that without error response will be always not nil.

https://storjlabs.atlassian.net/browse/SM-590

Change-Id: I0fc9539e429110a660eb28725b266d5e4771d198
 2020-04-02 12:20:19 +00:00
Egon Elbre
8f73fb7a32 all: simplify uuid usage 
uuid.UUID implements driver.Value so it can be directly used as a
scannable result.

Replace uses of dbutil.BytesToUUID with uuid.FromBytes.

Change-Id: I51a670185ceb3cc2199d5aa2b76bc3fc191ca8fe
 2020-04-02 05:48:58 +00:00
Yingrong Zhao
f663906357 storagenode/contact: call return value from mon.Task() on function finish 
Change-Id: I5e6462acb99ac1d28b5d2518d5db8a4afe593d11
 2020-04-01 23:26:14 +00:00
Egon Elbre
90319dbec1 scripts: fix test-sim-backwards 
Change-Id: I9d6644d4b493f5f6f60a960cacfaac2b5b828a5f
 2020-04-02 00:13:42 +02:00
Jeff Wendling
ffe7a3c211 satellite/compensation: make surge percent an int64 for strictcsv 
Change-Id: I1783bf73ee68ca9beb8a03f5928873fab0bbe95d
 2020-04-01 14:06:33 +00:00
Egon Elbre
644df8dcdc private/version: minimal fix for tag-release.sh 
Previous split to a storj.io/private repository broke tag-release.sh
script. This is the minimal temporary fix to make things work.

This links the build information to specified variables and sets them
inline. This approach, of course, is very fragile.

Change-Id: I73db2305e6c304146e5a14b13f1d917881a7455c
 2020-04-01 13:46:45 +00:00
NickolaiYurchenko
9a72b5fde9 web/storagenode: byte*h to byte*month conversion 
Change-Id: Iec95e0d6d946426df38e5725af3a889c6d81b23a
 2020-04-01 13:03:16 +00:00
Michal Niewrzal
d444fbadea scripts: cleanup rolling upgrade test 
* add script for easy rolling upgrade test local execution
* remove unneeded binaries building for rolling upgrade and versions
tests
* unify build process for Jenkins and local execution for rolling
upgrade and versions tests

Change-Id: Ic11211b83f3f447494bbd5827d2af77ea4b20dfe
 2020-04-01 12:30:08 +00:00
Kaloyan Raev
473cd4695f bump storj.io/uplink to v1.0.3 
Change-Id: I9466214260d139eed155c2d8fd725572cd5f0425
 2020-04-01 11:12:59 +00:00
Egon Elbre
a416b03941 satellite/accounting: fix TestProjectBandwidthTotal 
Test was inserting for past 4 days, however the test was summing up for
the current month.

Change-Id: I509afdc6a76b314a6bb90652ab70cd2c2bab1288
 2020-04-01 11:50:18 +03:00
Jeff Wendling
9bd0bd0c24 private/currency: add strictcsv support to microunit 
Change-Id: Iad2f6a07f189f2faa1d13bdb82dfa320921f6938
 2020-03-31 14:57:04 -06:00
Egon Elbre
0a69da4ff1 all: switch to storj.io/common/uuid 
Change-Id: I178a0a8dac691e57bce317b91411292fb3c40c9f
 2020-03-31 19:16:41 +03:00
Egon Elbre
1d79228ed0 satellite/metainfo: support uplink useragent 
Adds support for parsing user agent and specifying uplink version and
it's library dependencies.

Change-Id: Ibaddde4deb93e153ac05c91b676c5b5f1ae1aa37
 2020-03-31 15:11:31 +00:00
Qweder93
1fb37915c3 heldamount date picking hot fix 
Change-Id: I0cd0787e20e8af35a44bf05fa5c177434f563572
 2020-03-31 15:26:26 +03:00
Qweder93
dc32f1da55 storagenode/cache/heldamount added, errNoRows ignored 
Change-Id: If6b675e622d6c1324c0893c43cca93dc5323cd78
 2020-03-31 11:35:58 +00:00
Michal Niewrzal
0374e30678 scripts: fix storj-sim installation function for rolling upgrade tests 
Change-Id: Ifb749e0a41c57b90cc03293c2588b2ceb6a0a15a
 2020-03-31 09:11:00 +00:00
Jeff Wendling
e2ff2ce672 satellite: compensation package and commands 
Change-Id: I7fd6399837e45ff48e5f3d47a95192a01d58e125
 2020-03-30 14:08:14 -06:00
littleskunk
23e5a0471f
satellite/audit: clean up logging (#3832) 
Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
 2020-03-30 12:09:50 -06:00
Jess G
a73416e208
revert to use the config for storagesync (#3837) 
Change-Id: I11920b54736e7e548f5638a9ba5f6380a94d99c4
 2020-03-30 13:43:38 -04:00
Jennifer Johnson
d77f3b8786 satellitedb/migrate: set vetted_at backfill to now.day 
Change-Id: Ib2b12be43dbd3f3705b1891bc703ae15abb75e09
 2020-03-30 16:50:23 +00:00
Egon Elbre
439aba922a satellite/overlay: reduce overhead of GetNodes 
Instead of filtering on the client side it's better to filter on the
database side.

Change-Id: I845fbbe5ed28c2ffdb0b8a3f789b59c094fd1069
 2020-03-30 18:36:23 +03:00
Egon Elbre
cb781d66c7 satellite/overlay: optimize FindStorageNodes 
Reduce the number of fields returned from the query.

Benchmark results in `satellite/overlay`:

benchstat before.txt after2.txt
name                               old time/op  new time/op  delta
SelectStorageNodes-32              7.85ms ± 1%  6.27ms ± 1%  -20.18%  (p=0.002 n=10+4)
SelectNewStorageNodes-32           8.21ms ± 1%  6.61ms ± 0%  -19.53%  (p=0.002 n=10+4)
SelectStorageNodesExclusion-32     17.2ms ± 1%  15.9ms ± 1%   -7.55%  (p=0.002 n=10+4)
SelectNewStorageNodesExclusion-32  17.8ms ± 2%  16.1ms ± 0%   -9.38%  (p=0.002 n=10+4)
FindStorageNodes-32                48.4ms ± 1%  45.1ms ± 0%   -6.69%  (p=0.002 n=10+4)
FindStorageNodesExclusion-32       79.2ms ± 1%  76.1ms ± 1%   -3.89%  (p=0.002 n=10+4)

Benchmark results from `satellite/overlay` after making them parallel:

benchstat before-parallel.txt after2-parallel.txt
name                               old time/op  new time/op  delta
SelectStorageNodes-32               548µs ± 1%   353µs ± 1%  -35.60%  (p=0.029 n=4+4)
SelectNewStorageNodes-32            562µs ± 0%   368µs ± 0%  -34.51%  (p=0.029 n=4+4)
SelectStorageNodesExclusion-32     1.02ms ± 1%  0.84ms ± 0%  -18.08%  (p=0.029 n=4+4)
SelectNewStorageNodesExclusion-32  1.03ms ± 1%  0.86ms ± 2%  -16.22%  (p=0.029 n=4+4)
FindStorageNodes-32                3.11ms ± 0%  2.79ms ± 1%  -10.27%  (p=0.029 n=4+4)
FindStorageNodesExclusion-32       4.75ms ± 0%  4.43ms ± 1%   -6.56%  (p=0.029 n=4+4)

Change-Id: I1d85e2764eb270f4c2b1998303ccfc1179d65b26
 2020-03-30 18:36:23 +03:00
Stefan Benten
028ce16020
cla: Adding sixcorners to the CLA approved list (#3836) 2020-03-30 18:33:06 +03:00
Egon Elbre
a6540dc3ef satellite/overlay: remove unused KeyLock 
Change-Id: Ie99f97772824ceafb3d97453545fc6e96be2fb6f
 2020-03-30 16:47:48 +03:00
Ivan Fraixedes
de903a652d satellite/accounting: Test no repair traffic in billing 
Add a test for checking that the billing doesn't include traffic due to
audits and repairs.

Change-Id: I1ce1199ec87c4440082c42b100124aeb14200f41
 2020-03-30 10:48:35 +00:00
littleskunk
048ca4558f
satellite/repair: clean up logging (#3833) 
Co-authored-by: Michal Niewrzal <michal@storj.io>
 2020-03-30 11:59:56 +02:00
Yingrong Zhao
831668478a scripts/tests: fix gateway installation 
Change-Id: I1629644f543505c27d4edb8f7bbe97d037cdc8a8
 2020-03-29 19:24:05 +00:00
NickolaiYurchenko
4122be154e web/storagenode: payout page adaptation 
Change-Id: Iaa9fc69567cc6a3daf11829d0434f12098098f5b
 2020-03-29 18:39:30 +00:00