Commit Graph

303 Commits

Author SHA1 Message Date
Stefan Benten
0b43b93259 satellite/satellitedb: make limits per default NULL
This change completes the column migration of
5f6fccc6e8 and
2f648fd981.
It resets every users project limits who are below or equal to our
current production defaults.

Change-Id: Ie041d08bb67b62844f6023190fc00bc2dad5b1cb
2020-10-14 20:28:16 +00:00
Jessica Grebenschikov
4a2c66fa06 satellite/accounting: add cache for getting project storage and bw limits
This PR adds the following items:
1) an in-memory read-only cache thats stores project limit info for projectIDs

This cache is stored in-memory since this is expected to be a small amount of data. In this implementation we are only storing in the cache projects that have been accessed. Currently for the largest Satellite (eu-west) there is about 4500 total projects. So storing the storage limit (int64) and the bandwidth limit (int64), this would end up being about 200kb (including the 32 byte project ID) if all 4500 projectIDs were in the cache. So this all fits in memory for the time being. At some point it may not as usage grows, but that seems years out.

The cache is a read only cache. When requests come in to upload/download a file, we will read from the cache what the current limits are for that project. If the cache does not contain the projectID, it will get the info from the database (satellitedb project table), then add it to the cache.

The only time the values in the cache are modified is when either a) the project ID is not in the cache, or b) the item in the cache has expired (default 10mins), then the data gets refreshed out of the database. This occurs by default every 10 mins. This means that if we update the usage limits in the database, that change might not show up in the cache for 10 mins which mean it will not be reflected to limit end users uploading/downloading files for that time period..

Change-Id: I3fd7056cf963676009834fcbcf9c4a0922ca4a8f
2020-09-25 16:28:49 +00:00
Stefan Benten
5f6fccc6e8 satellite/satellitedb: makes limits nullable change backwards compatible
Our current endpoints bail on us, if the column data is null. Thus we need
to take the intermediate step and set the default to a fixed value and
reset those with the following release.

It sets the default column value to our current config values of 50GB
for storage and bandwidth and 100 buckets, while still enabling the field to be nullable.
All 0 values are migrated to be the default as well to ensure they can
keep using their projects, as with the original change, 0 actually means 0.

Change-Id: I797be80ce2d2105091599dc1b3fc76f74336b66b
2020-09-23 17:54:42 +02:00
Stefan Benten
2f648fd981 satellite: make limits be nullable
Currently we have no way to actually set one
of the following limits to 0 (meaning not usable):

- maxBuckets
- usageLimit
- bandwidthLimit

With having the field nullable,
NULL corresponds to the global default,
0 now actually 0 and
a set value determines a custom limit.

Change-Id: I92bb77529dcbd0881ae8368921be9d246eb0919e
2020-09-21 19:34:19 +00:00
Cameron Ayer
e7c34a053d satellite/satellitedb: add column and index "updated_at" to injuredsegments
Change-Id: I59e9bb2077885f09e17795375fe98ed31bd83d54
2020-09-14 12:53:04 -04:00
Moby von Briesen
2d01dd9732 satellite/satellitedb: Add online_score column to nodes table
Add online score used for the new audit history offline tracking system
to the nodes table. This allows us easy access to the node's online
score for the storagenode dashboard as well as for data analysis.

Change-Id: Ie99be1192e5236862a5b3dbed2e5ef03b9169410
2020-08-31 15:07:07 +00:00
Egon Elbre
94a09ce20b all: add missing dots
Change-Id: I93b86c9fb3398c5d3c9121b8859dad1c615fa23a
2020-08-11 17:50:01 +03:00
Kaloyan Raev
7552ff26ec satellite/db: drop project_invoice_stamps table
It's an obsolete table from earlier state of Stripe invoices
implementation. No code is currently using it. It is confirmed that this
table is currently empty across all satellites.

Change-Id: I12d2756578faf8418ea8f3b09088e885694b8925
2020-08-10 13:22:10 +00:00
Kaloyan Raev
edfd3d7661 satellite/payments: delete credits and credits_spendings db tables
Jira: https://storjlabs.atlassian.net/browse/USR-822

This the last step of dropping these 2 db tables. It also deletes all
code associate with them.

Change-Id: I8be840dc2a7be255cf6308c9434b729fe4d9391e
2020-07-30 12:19:57 +03:00
Egon Elbre
36ed939b89 satellite/orders: add buckets db to service
We need to add bucket UUID into the order limit,
hence we need access to the buckets table.

Change-Id: I348ce1f709c9fcdec5c4034acaab59805b33da9f
2020-07-24 17:36:49 +03:00
Egon Elbre
080ba47a06 all: fix dots
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
2020-07-16 14:58:28 +00:00
stefanbenten
0209a2095f satellite/{console,satellitedb}: add project_limit column to users table
Change-Id: I603f085f17ca5b413dd1c6837c2081f9e7e791a1
2020-07-15 17:27:31 +00:00
Jennifer Johnson
784a156eea satellite: prevents uplink from creating a bucket once it exceeds the max bucket allocation.
Change-Id: I4b3822ed723c03dbbc0df136b2201027e19ba0cd
2020-07-15 17:27:05 +00:00
stefanbenten
0a32ba0e6b satellite/admin: add project rename functionality
Change-Id: I4c0f42d4c2c26859279f247f94cef97a8ff630a9
2020-07-14 11:36:49 +00:00
Jessica Grebenschikov
8abb907010 satellite/orders: add settle orders with window
Why: We need a way to cut down on database traffic due to bandwidth
measurement and tracking.

What: This changeset is the Satellite side of settling orders in 1 hr windows.
See design doc for more details: https://review.dev.storj.io/c/storj/storj/+/1732

Change-Id: I2e1c151e2e65516ebe1b7f47b7c5f83a3a220b31
2020-07-13 15:41:29 -07:00
paul cannon
bbdb351e5e all: use jackc/pgx in place of lib/pq
What:

Use the github.com/jackc/pgx postgresql driver in place of
github.com/lib/pq.

Why:

github.com/lib/pq has some problems with error handling and context
cancellations (i.e. it might even issue queries or DML statements more
than once! see https://github.com/lib/pq/issues/939). The
github.com/jackx/pgx library appears not to have these problems, and
also appears to be better engineered and implemented (in particular, it
doesn't use "exceptions by panic"). It should also give us some
performance improvements in some cases, and even more so if we can use
it directly instead of going through the database/sql layer.

Change-Id: Ia696d220f340a097dee9550a312d37de14ed2044
2020-07-13 15:54:41 +00:00
Jeff Wendling
885ef70c58 satellite/nodeapiversion: new table for tracking node api usage
This system tracks an abstract "api version" from nodes based on
their usage, allowing us to have latching behavior where if a node
ever uses a new api, it can be blocked from using the old api.

This is better than using self-reported semver version information
because the node cannot lie, there's no confusion about what semver
version implies which features, no questions about dev and ci
environments, and no dependencies between reporting the version
and using the new api.

Change-Id: Ifeced5c9ae8e0a16102d79635e176a7d3bdd8ed4
2020-07-09 15:02:25 +00:00
Cameron Ayer
e3088d9ad5 satellite/satellitedb: add new DB table audit_histories
Change-Id: I5f854514994cab9a68cf978f2dabfb588df695f5
2020-07-01 21:14:35 +00:00
Isaac Hess
2d727bb14e satellite: Check macaroon revocation
When a request comes in on the satellite api and we validate the
macaroon, we now also check if any of the macaroon's tails have been
revoked.

Change-Id: I80ce4312602baf431cfa1b1285f79bed88bb4497
2020-06-22 13:50:07 -06:00
Cameron Ayer
0885ba5646 satellite/satellitedb: add new columns for offline suspension
add new columns `offline_suspended` and `under_review` to nodes table.
`unknown_audit_suspended` is a new column which will replace `suspended`

Change-Id: I22ddeb338ea0ff63f14332a7ebd0f3e9e4c06cdc
2020-06-15 04:00:20 +00:00
Egon Elbre
36c461bd59 private/tagsql: track proper closing of rows and statements
This ensures that rows are closed to avoid leaks.
Also verifies that Err() is called, to ensure that no
error is left behind.

Change-Id: Idd1bec9bf479f40021da67b2c80ce83033149469
2020-06-05 18:25:43 +00:00
Michal Niewrzal
b20ced9519 satellite/satellitedb: drop project_id column from coupons table
This is last part of https://storjlabs.atlassian.net/browse/USR-818

Change-Id: I053d11b37df962c12e46645bae2fc2dad49c9755
2020-06-03 14:56:41 +00:00
Kaloyan Raev
49571f1a23 satellite/payments: all invoice commands require period
To avoid including multiple months in a single invoice, we need all
inspector's invoice commands to run in for specific period.

See https://storjlabs.atlassian.net/browse/USR-725

Change-Id: I3637dc189234f02350daca8d897c21765762ea55
2020-05-14 11:50:19 +00:00
Stefan Benten
e23bd806b4
satellite/accounting: separate usage and bandwidth limit (#3878) 2020-05-12 15:01:15 +02:00
Ethan
acf53bea4d satellite/orders;accounting: Add monthly project download bandwidth rollup
See https://storjlabs.atlassian.net/browse/SM-776

Change-Id: Ifd5cccea43c556fd59822d17344f399cfe9a7164
2020-05-04 15:49:57 +00:00
Ethan Adams
60e07f0a8b Revert "satellite/accounting: Remove unnecessary index bucket_bandwidth_rollups_project_id_action_interval_index"
This reverts commit 105dc7acc6.

Reason for revert: Recent changes to the Postgres query plan seems to want to use this index now.  Reverting until we have time to analyze what's happening.

Change-Id: I74b4b5a8f15c3850d8a958a29f51dbc80e7c282c
2020-04-22 14:49:04 +00:00
Ethan
105dc7acc6 satellite/accounting: Remove unnecessary index bucket_bandwidth_rollups_project_id_action_interval_index
See https://storjlabs.atlassian.net/browse/SM-738

Change-Id: I9ba3cc3fbff9f13fc0b95d25feee5a19e5a5c486
2020-04-21 16:43:09 +00:00
Ethan
4cd86ff780 satellite/accounting: Add index on bucket_bandwidth_rollups for action, interval_start, and project_id
See https://storjlabs.atlassian.net/browse/SM-551 for details

Change-Id: I104c4e87d5aef500cc4a3893817763808f76c484
2020-04-17 19:14:45 +00:00
Jeff Wendling
2ded64ba2c satellite/compensation: more fixes to get prod running smoothly
Change-Id: I13a76d9d49222fb10796415a015f224d4084fde3
2020-04-07 10:10:27 +00:00
Jennifer Johnson
1547e791a3 satellitedb: remove free_bandwidth column from nodes table
Change-Id: I9d1d3de9216c6533c1042ef473631721a011d086
2020-04-06 09:30:28 +00:00
Ethan
df462d7265 satellite/accounting: Add index on bucket_bandwidth_rollups to minimize full table scans
https://storjlabs.atlassian.net/browse/SM-545

Change-Id: I5599a72a991d70236f17beca027e9bc032777177
2020-03-26 19:53:50 +00:00
Jessica Grebenschikov
aeab599d21 satellitedb: removed unused id on storagenode_storage_tallies table, add index on node_id
The goal of this change is to improve the storagenode_storage_tallies table by removing the unneeded id column that is not being used but only taking up space, and also to add an index on a different column that needs it. Removing and adding a column seems simple, but ended up being more complicated because of some cockroachdb limitations.

The cockroachdb limitation when trying to remove a column from a table and create a new primary key are:
1. only allows primary key creation at table creation time (docs: https://www.cockroachlabs.com/docs/stable/primary-key.html)
2. table drop or rename is performed async and cannot be done in a transaction (issue: https://github.com/cockroachdb/cockroach/issues/12123, https://github.com/cockroachdb/cockroach/issues/22868)

To address these differences between cockroachdb  and Postgres, this PR performs different migrations for the two database. The Postgres migration is straight forward and what you would expect, but the cockroach migration has two main changes:
1. To change a primary key, use the recommended process from the cockroachdb docs to create a new table with the new primary key you want and then migrate the data.
2. In order to do 1, we needed to do the new table renaming in a separate transaction from the data migration.

Ref: SM-65

Change-Id: Idc9aee3ab57aa4d5570e3d2980afea853cd966bf
2020-03-20 14:39:44 -07:00
Jennifer Johnson
9b78473c0c satellitedb: adds vetted_at nullable timestamp to nodes table
Change-Id: I42d5a396b4eecbad26b683c6aee51e043d2ff034
2020-03-20 01:37:28 +00:00
Moby von Briesen
2f991b6c56 satellite/{overlay, satellitedb}: account for suspended field in overlay cache
Make sure that suspended nodes are treated appropriately by the overlay
cache. This means we should expect the following behavior:
* suspended nodes (vetted or not) should not be selected for uploading
new segments
* suspended nodes should be treated by the checker and repairer as
"unhealthy", and should be removed upon successful repair

This commit also removes unused overlay functionality.

Fixes a bug with commit 8b72181a1f where
the audit reporter was automatically suspending nodes regardless of
audit outcome (see test added).

Tests:
* updates repair tests to ensure that a suspended node is treated as
unhealthy and will be removed from the pointer on successful repair
* updates overlay tests for KnownUnreliableOrOffline and KnownReliable
to expect suspended nodes to be considered "unreliable"
* adds satellitedb test that ensures overlay.SelectStorageNodes and
overlay.SelectNewStorageNodes do not include suspended nodes
* adds audit reporter test to ensure that different audit outcomes
result in the correct suspended/disqualified states

Change-Id: I40dba67278c8e8d2ce0bcec5e0a5cb6e4ce2f561
2020-03-17 17:14:56 +00:00
Jeff Wendling
41887883f3 satellite/satellitedb: check indexes on migration
Change-Id: I5ba7ae2b512d77c70405ce332158f12128e27eed
2020-03-13 10:45:22 +00:00
Jessica Grebenschikov
803e2930f4 satellite: use IP for all uplink operations, use hostname for audit and repairs
My understanding is that the nodes table has the following fields:
- `address` field which can be a hostname or an IP
- `last_net` field that is the /24 subnet of the IP resolved from the address

This PR does the following:
1) add back the `last_ip` field to the nodes table
2) for uplink operations remove the calls that the satellite makes to `lookupNodeAddress` (which makes the DNS calls to resolve the IP from the hostname) and instead use the data stored in the nodes table `last_ip` field. This means that the IP that the satellite sends to the uplink for the storage nodes could be approx 1 hr stale. In the short term this is fine, next we will be adding changes so that the storage node pushes any IP changes to the satellite in real time.
3) use the address field for repair and audit since we want them to still make DNS calls to confirm the IP is up to date
4) try to reduce confusion about hostname, ip, subnet, and address in the code base

Change-Id: I96ce0d8bb78303f82483d0701bc79544b74057ac
2020-03-11 09:11:40 -07:00
Bill Thorp
e99e675fb1 satellite/satellitedb: use time zones with all timestamps
The migration was broken into one migration per table to reduce table locking and reduce the
chances of failure due to SQL timeouts.

Of the 14 fields that lacked time zones, only the 3 named 'interval_start` seemed to have non-UTC data in them.
These fields are fixed in the migration by removing the +00 and adding  AT TIME ZONE current_setting('TIMEZONE')
Field with good data are migrated by adding AT TIME ZONE 'UTC'

Note that postgres's timezone() is different than cockroach's timezone() so AT TIME ZONE is used.

https://storjlabs.atlassian.net/browse/SM-104

Change-Id: I410f2f1d7c11b143f17844347f37e6f4b1e70fce
2020-03-05 21:11:25 +00:00
Moby von Briesen
f495544c56 satellite/satellitedb/dbx: add fields to node table for placing nodes into suspended mode for too many unknown-error audits
Change-Id: Iac9a619e5c08377de87ffdf4acdd0155027f5eb3
2020-03-03 03:30:59 +00:00
Jeff Wendling
1db087cfba satellite/satellitedb: migration to create tables for compensation
these tables are used in future commits with respect to the new
storagenode payments code. if we create them now, it will make
backfilling them with historical data easier.

Change-Id: I3c08c9770ec5b2baa38b4f2fd18c2f07746a61c2
2020-02-27 17:34:50 +00:00
Moby von Briesen
4e5a7f13c7 satellite/repair/queue: Prioritize selection of items off repair queue by segment health
Add a column to the repair queue table in the satellite db for healthy
piece count. When an item is selected from the repair queue, the least
durable segment that has not been attempted in the past hour should be
selected first. This prevents our repairer from getting stuck doing work
on segments that are close to the repair threshold while allowing
segments that are more unhealthy to degrade further.

The migration also clears the repair queue so that the migration runs
quickly and we can properly account for segment health in future repair
work.

We do not select items off the repair queue that have been attempted in
the past six hours. This was changed from on hour to allow us time to
try a wider variety of segments when the repair queue is very large.

Change-Id: Iaf183f1e5fd45cd792a52e3563a3e43a2b9f410b
2020-02-26 09:54:16 -05:00
Jeff Wendling
f671eb2beb satellite/satellitedb: use queue for orders to get back fast billing
This change adds two new tables to process orders as fast as we used
to but in an asynchronous manner and with hopefully less storage
usage. This should help scale on cockroach, but limits us to one
worker. It lays the groundwork for the order processing pipeline to
be queue rather than database driven.

For more details, see the added fast billing changes blueprint.

It also fixes the orders db so that all the timestamps that are
passed to columns that do not contain a time zone are converted to
UTC at the last possible opportunity, making it less likely to use
the APIs incorrectly. We really should migrate to include timezones
on all of our timestamp columns.

Change-Id: Ibfda8e7a3d5972b7798fb61b31ff56419c64ea35
2020-02-24 17:07:07 +00:00
Jeff Wendling
2d2f5e1a7f satellite/satellitedb/dbx: remove typo in dbx file and format it
Change-Id: I756315d6228ac9edd35cad8b496d36ecf2b5d26f
2020-02-11 14:15:13 -07:00
Qweder93
dc075eaa96 satellite/payments : deposit bonuses (credits) added
Change-Id: Ib151bbb9b02d655fa619c53bfbc04ed6f3bb39e0
2020-02-11 11:11:42 +00:00
Jeff Wendling
7999d24f81 all: use monkit v3
this commit updates our monkit dependency to the v3 version where
it outputs in an influx style. this makes discovery much easier
as many tools are built to look at it this way.

graphite and rothko will suffer some due to no longer being a tree
based on dots. hopefully time will exist to update rothko to
index based on the new metric format.

it adds an influx output for the statreceiver so that we can
write to influxdb v1 or v2 directly.

Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2020-02-05 23:53:17 +00:00
Jeff Wendling
d20db90cff private/dbutil/txutil: create new transactions for retries
it was noticed that if you had a long lived transaction A that
was blocking some other transaction B and A was being aborted
due to retriable errors, then transaction B was never given
priority. this was due to using savepoints to do lightweight
retries.

this behavior was problematic becaue we had some queries blocked
for over 16 hours, so this commit addresses the issue with two
prongs:

    1. bound the amount of time we will retry a transaction
    2. create new transactions when a retry is needed

the first ensures that we never wait for 16 hours, and the value
chosen is 10 minutes. that should be long enough for an ample
amount of retries for small queries, and huge queries probably
shouldn't be retried, even if possible: it's more preferrable to
find a way to make them smaller.

the second ensures that even in the case of retries, queries that
are blocked on the aborted transaction gain priority to run.

between those two changes, the maximum stall time due to retries
should be bounded to around 10 minutes.

Change-Id: Icf898501ef505a89738820a3fae2580988f9f5f4
2020-02-01 18:34:28 +00:00
Jeff Wendling
d09bd4a749 satellite/satellitedb/dbx: regenerate with paged composite key fixes
before dbx would generate a compilcated blob of conditions that
encoded a row comparison, which only optimized to an index seek
on cockroachdb. this means that sqlite and postgres both had
quadratic behavior on paged queries of this form. instead, use
the implicit row construction feature supported in all of the
databases to do paged support so that they all optimize well.

Change-Id: Iac8703929ba2a59ee3ffa619b916d12663422887
2020-01-27 12:43:16 -07:00
ccase
a9e4c6f66d
satellite/satellitedb/dbx: Remove bashism from gen.sh
Change-Id: Ia698edae99d7ff0c73fa457b4a3c0a7b5f0bbec5
2020-01-23 17:09:07 -05:00
Ethan
21a5d70a83 satellite/metainfo: Rate limiting - API requests
Limits how many times metainfo APIs can be called per second by project ID. If limit is exceeded, the API will return Unauthorized/Too Many requests.

Limit per second and the size of the limiter cache per project are configurable, as well as whether the limiter is enabled.

Tests added/updated for the new rate_limit field in projects table.
Tests added for exceeding limits and disableing limiter.

Change-Id: Ic8ad102de3b690a475809d4f684156d5715f20fa
2020-01-21 14:25:04 +00:00
Egon Elbre
c1c878efcf all: fix import groupings
check-imports was broken and didn't complain about things.

Change-Id: I38adafd16b4aba86f0eb4f53427b4393f9a6c710
2020-01-20 17:47:44 +00:00
Egon Elbre
1abfe42142 satellite: use tagsql
Change-Id: I2170dee409fb0c2fe85913ddd36e7811a3b853ed
2020-01-19 14:39:16 +02:00
Isaac Hess
cd48dc369a satellite/satellitedb: Remove unused indexes
Change-Id: I875b94574eacf9d2df537bcf1f42f30e0bf60ab9
2020-01-16 16:06:21 -07:00
Jeff Wendling
47bb7a7a86 satellite/satellitedb/dbx: regenerate with default support
Change-Id: I0dab34f27af913795ef95ef92173844c2f53b29b
2020-01-16 22:13:57 +00:00
Jeff Wendling
696d98a232 satellite/satellitedb: fix nitpicks and timestamp issue found in review
warning: databases migrated to version 77 before this commit
is merged must be manually re-migrated. this should not be a
problem for anything but staging databases.

Change-Id: Ie1631c48379472352014183ee43f1465e22200f7
2020-01-16 21:22:38 +00:00
Jeff Wendling
78c6d5bb32 satellite/satellitedb: reported_serials table for processing orders
this commit introduces the reported_serials table. its purpose is
to allow for blind writes into it as nodes report in so that we have
minimal contention. in order to continue to accurately account for
used bandwidth, though, we cannot immediately add the settled amount.
if we did, we would have to give up on blind writes.

the table's primary key is structured precisely so that we can quickly
find expired orders and so that we maximally benefit from rocksdb
path prefix compression. we do this by rounding the expires at time
forward to the next day, effectively giving us storagenode petnames
for free. and since there's no secondary index or foreign key
constraints, this design should use significantly less space than
the current used_serials table while also reducing contention.

after inserting the orders into the table, we have a chore that
periodically consumes all of the expired orders in it and inserts
them into the existing rollups tables. this is as if we changed
the nodes to report as the order expired rather than as soon as
possible, so the belief in correctness of the refactor is higher.

since we are able to process large batches of orders (typically
a day's worth), we can use the code to maximally batch inserts into
the rollup tables to make inserts as friendly as possible to
cockroach.

Change-Id: I25d609ca2679b8331979184f16c6d46d4f74c1a6
2020-01-15 19:21:21 -07:00
Jeff Wendling
9da16b1d9e satellite/satellitedb/dbx: name the package dbx
everyone was importing it as dbx anyway. why should it be
named satellitedb? so yeah just pass the "-p dbx" flag.

Change-Id: I5efa669f4f00f196b38a9acd0d402009475a936f
2020-01-15 15:16:39 -07:00
Egon Elbre
64fb2d3d2f Revert "dbutil: statically require all databases accesses to use contexts"
This reverts commit 8e242cd012.

Revert because lib/pq has known issues with context cancellation.
These issues need to be resolved before these changes can be merged.

Change-Id: I160af51dbc2d67c5449aafa406a403e5367bb555
2020-01-15 07:28:00 +00:00
JT Olio
8e242cd012 dbutil: statically require all databases accesses to use contexts
this will allow for some nice runtime analysis down the road.
also, this allows for wrapping database handles in a way that
can interact with these contexts

requires https://review.dev.storj.io/c/storj/dbx/+/514

Change-Id: Ib087b7cd73296dd2c1e0331314da34d861f61d2b
2020-01-14 18:20:47 -05:00
crawter
a57ce18f58 satellite/payments: coupons, coupons usage, invoice generation with pricing model applied
Change-Id: Ic5d5a2fc116388647efe46896cfccc2038c77537
2020-01-14 12:45:00 +00:00
littleskunk
bcc23f6869
Satellite/orders: remove allocated bandwith from storagenode_bandwidth_rollups
When an uplink requests an upload or download from the satellite we are trackig the
allocated bandwidth twice. The value in bucket_bandwidth_rollups is used
for project limits but the value in storagenode_bandwidth_rollups is not
used at all. We can increase the performance by removing it. Uplinks
will get a faster response from the satellite.

Change-Id: Icccd41f94107ef34668f30f99bf5f728c384b07e
2020-01-12 16:20:47 +01:00
Egon Elbre
082ec81714
uplink: move to storj.io/uplink (#3746) 2020-01-08 15:40:19 +02:00
Jeff Wendling
c740b82e66 satellitedb/dbx: remove sed usage for bash script
turns out portable sed is hard: it has to work with both
linux and bsd sed, etc. instead, use a really really basic
bash script and a temporary file. this should be much less
likely to cause issues on a wide range of machines.

Change-Id: Ia759789fb52aa1ee3361426bb6c02ed4eac3d23a
2020-01-08 01:40:24 +00:00
paul cannon
6b21334c47 satellite/satellitedb: use txutil.ExecuteInTx in dbx WithTx()
Change-Id: I42ec21fdf117c661b3e1687a04014650c3a6ab97
2020-01-07 17:00:08 -06:00
Moby von Briesen
6c2e4cc0cd satellite/overlay: Return NodeLastContact instead of a node dossier from
overlay.GetOfflineNodesLimited

We only care about node ID, address, and last contact success/failure
from the downtime service, so the overlay should only return these
values for the downtime-specific queries.

Change-Id: I08a6ecfdd2a12b82cae62e87d6adeab53975bfce
2020-01-06 17:12:30 -05:00
paul cannon
a33734bee7 satellite/satellitedb/dbx: add cockroach driver type
Change-Id: I7a0da6e066c67a521fc1b23b085ab8554eee0d4c
2020-01-06 18:01:03 +00:00
Ethan
05b406e992 satellite:{downtime,overlay}: Implement offline node detection chore
https://storjlabs.atlassian.net/browse/V3-3398

Change-Id: I598c3bad819026377d1d113c099dc9bba8b02742
2020-01-03 17:10:03 +00:00
Moby von Briesen
ff74b44c5f satellite/overlay: Add ability for overlay to get offline nodes ordered by last checked time
This is required for the downtime tracking service: https://storjlabs.atlassian.net/browse/V3-2545

Change-Id: I286cdc07d802393948eb10c25c45ba78cc3ceafc
2020-01-02 16:39:38 +00:00
Moby von Briesen
bb3baf5a4e satellite/satellitedb: Add nodes_offline_times table for downtime tracking
Change-Id: If6b80fe0a20d88cedacaf4b76b75aa21d0af2465
2019-12-30 15:45:02 -05:00
Jeff Wendling
ffbc43d170 satellite/satellitedb/dbx: monitor the database calls
Change-Id: I9fbdc4a35fedfc7a1b4e1b630d2a3664b3218e67
2019-12-18 21:57:36 +00:00
Jessica Grebenschikov
c5116cb2a0 satellitedb: fix migration cockroach test
Change-Id: Ie3b4a4b0795d156238d50a58078282cc0918a334
2019-12-16 18:02:31 +00:00
Yaroslav Vorobiov
8cf1aa6e4f
satellite/accounting: fix project limits migration (#3717) 2019-12-10 18:12:49 +02:00
Yehor Butko
756b9b9e2b
satellite/payments: coupons and coupon usage (#3648) 2019-11-26 19:58:51 +02:00
Yaroslav Vorobiov
8a002e8c8e
satellite/accounting: separate project limit from project entity (#3632) 2019-11-25 16:18:04 +02:00
Matt Robinson
9af97d366a Make sed a little more cross platformable (#3629) 2019-11-22 11:17:02 -07:00
Yaroslav Vorobiov
53c6741ba6
satellite/payments: add API for retrieving conversion ratio, convert tokens to USD before applying to balance (#3530) 2019-11-15 16:59:39 +02:00
Yehor Butko
a8e4e9cb03
satellite/payments: project usage charges (#3512) 2019-11-15 16:27:44 +02:00
Egon Elbre
1e64006e32 lint: add staticcheck as a separate step (#3569) 2019-11-14 10:31:30 +02:00
Natalie Villasana
1a9757a7f2 satellite/gracefulexit: add count for order limits sent from satellite to exiting node (#3544) 2019-11-13 09:54:50 -05:00
Yaroslav Vorobiov
36311a3a05
satellite/console: add token deposit API, populate billing history with transactions (#3500) 2019-11-12 13:14:34 +02:00
Ethan Adams
f3dccb56b1
satellite/gracefulexit: Check if pointer has been overwritten or deleted before sending transfer message. (#3481) 2019-11-07 11:13:05 -05:00
Yaroslav Vorobiov
35edc2bcc3 satellite/payments: invoice creation (#3468) 2019-11-05 15:16:02 +02:00
Yehor Butko
0c2e498f09
satellite/satellitedb: console tables archview comments updated (#3465) 2019-11-04 16:37:39 +02:00
Yaroslav Vorobiov
30a3205745
satellite/payments: update account balance (#3379) 2019-10-29 18:04:34 +02:00
Egon Elbre
2e44a9fa6b
satellite/satellitedb/dbx: remove sqlite.sql file (#3409) 2019-10-29 12:35:27 +02:00
Ethan Adams
9905f2c61e add piece num to transfer queue PK (#3390) 2019-10-28 11:08:33 -04:00
Yaroslav Vorobiov
1a304f5ef9
satellite/payments: add payments loop, update pending transactions (#3318) 2019-10-23 15:04:54 +03:00
Egon Elbre
3c438f31bd
satellite/satellitedb: remove sqlite support (#3296) 2019-10-19 00:27:57 +03:00
Natalie Villasana
45c35d7c3f
satellite/satellitedb: add exit_status column to nodes table (#3301) 2019-10-17 11:01:39 -04:00
Yaroslav Vorobiov
24e72f35d3
satellite/payments: token deposit (#3283) 2019-10-17 17:04:50 +03:00
Yehor Butko
a5f4bbee22
satellite/payments: dbx scheme renamed, userID placed on Account level (#3281) 2019-10-15 21:05:45 +03:00
Ethan Adams
a1275746b4
satellite/gracefulexit: Implement the 'process' endpoint on the satellite (#3223) 2019-10-11 17:18:05 -04:00
Yehor Butko
451909b3ec
satellite/payments: account balance (#3242) 2019-10-11 18:00:35 +03:00
Yehor Butko
0cc23add5b
satellite/payments - payment account setup (#3187) 2019-10-10 20:12:23 +03:00
Bogdan Artemenko
5f775b9e46
satellite/console: Added error for adding api key with existing name attempt. (#3185) 2019-10-10 16:28:35 +03:00
Ethan Adams
9edfb6efe0
satellite/satellitedb: Initial GE Satellite DB Implementation (#3049)
Initial GE Satellite DB impl
Add basic CRUD operations for graceful_exit_progress and graceful_exit_transfer_queue tables.
2019-09-25 11:12:44 -06:00
Ethan Adams
886041e0ba
satellite/satellitedb: add new graceful exit tables and add graceful exit fields to nodes (#3033)
DB schema changes for satellite to support Graceful Exit
2019-09-13 12:57:32 -04:00
Egon Elbre
8ef57a2af3
satellite/satellitedb: use noreturn (#3022) 2019-09-12 20:31:50 +03:00
Bogdan Artemenko
2c7813d40d
satellite/console: Added email normalization to users table (#2586) 2019-09-10 17:00:33 +03:00
Bryan White
a33106df1c
satellite/satellitedb: persist piece counts to/from db (#2803) 2019-08-27 14:37:42 +02:00
aligeti
33aff71959 satellitedb/overlay: add database for storing peer identities (#2764) 2019-08-26 19:49:42 +03:00
Bryan White
6400d63a6c
satellite/satellitedb: Add piece count column to nodes table (#2795) 2019-08-19 12:58:13 +02:00
aligeti
32f95a14fd
satellite/certdb: remove certdb that was used to store uplink certificates (#2760)
* satellitedb/certDB: refactors of the node certificate storage DB table

The existing implementation doesnt allow to store the complete certificate chain of uplinkIDs or storagenodeIDs, so the current table is dropped and new table will be added which addresses the storage and retrieval of certificates

pkg/identity: fixes spelling mistakes that I missed on PR#2754

Fixes V3-1992/V3-2388
2019-08-12 10:41:34 -04:00
Yehor Butko
b0e5964718
satellite/console: create owner_id column for project table (#2706)
* satellite/console: create owner_id column for project table
2019-08-07 15:28:13 +03:00
aligeti
65932ad692
Updating the certdb to support storage of multiple public keys for same node ID (#2692)
* updating the certdb to support storage of multiple public keys for same node ID
2019-08-02 13:47:35 -04:00
Yingrong Zhao
51833d0650
satellite/satellitedb: get active offer for partners (#2664)
* get partner offer

* fix lint

* fix deleting user account

* fix sqlite query

* add comments

* fix migration

* fix query_test

* add error logs

* add tests for user credits
2019-08-01 13:46:33 -04:00
Faris Huskovic
2144181c99
satellite/marketingweb: Add Partner Offer Type (#2540) 2019-07-25 18:06:23 -04:00
aligeti
29b576961f
value attribution merge fix and more test cases (#2588)
* value attribution merge fix and more test cases
2019-07-19 11:17:34 -04:00
Bill Thorp
a7cc940776
Nodes should not be able to fail the same audit multiple times (#2404)
update pointer on audit failure
2019-07-18 14:08:15 -04:00
Jess G
3af9250659
update irreparableDB.GetLimited query to use where instead of offset (#2585)
* update query to use where instead of offset, update tests

* update cmd/inspector irreparable

* add comment for offset
2019-07-18 09:21:21 -07:00
aligeti
daa3b32ee2
Add Attribution Columns to appropriate tables for OSPP referral link (#2516)
* adds "partner_id" column to user, project, api_key & bucket_metainfo tables
2019-07-12 13:59:19 -04:00
Maximillian von Briesen
b590e53d64
Order by attempted time in injured segments select (#2533) 2019-07-12 13:35:20 -04:00
Alexander Leitner
64b2769de3
discovery: parallelize refresh (#2535)
* parallelize discovery refresh

* add paginateQualifiedtest, address pr comments

* Remove duplicate uptime update

* Lower concurrency in Testplanet for discovery
2019-07-12 10:35:48 -04:00
Yingrong Zhao
d887ffec62
satellite/satellitedb: add default offer for offers table (#2522)
* add default offer for offers table

* fix migration test

* Trigger Jenkins

* set the default value to be correct type

* skip soon will deleted test

* fix test data

* add orderby for ListAll

* change durations, redeemable cap to be a nullable field

* remove unecessary code
2019-07-12 10:19:38 -04:00
Yingrong Zhao
6e57b102c5
satellite/satellitedb: remove num_redeemed in offers table (#2510)
* remove num_redeemed

* remove the field frol Offer struct

* remove the field in the UI templates

* fix merge conflict

* fix migration

* fix merge conflict

* fix typo
2019-07-12 09:19:18 -04:00
Jess G
f11bf46a11
Jg/1967 mv bucket metadata uplink (#2505)
* add bucketstore, add init uplink bucket

* update uplink to use bucket rpc

* fix tests

* wrap metainfo client errors

* add allowedBucket struct, fix tests

* update comment

* add paging

* updates per CR

* add test for pagination

* fix lint

* fix uplink test so its easier tyo understand

* fix gateway pagination bug

* changes per cr

* fix bug w/allowedBuckets, add test to catch
2019-07-12 08:57:02 -04:00
Michal Niewrzal
268c629ba8
Replace base64 encoding for path segments (#2345) 2019-07-11 13:26:07 -04:00
Bogdan Artemenko
32e0227c45
Project Payment methods (#2037) 2019-07-10 23:29:26 +03:00
Faris Huskovic
0d294103e9
satellite/rewards: nicer offers handling (#2390)
* organize offers

* revert changes to go.mod and go.sum

* change OfferStatus enums back to original

* revert modified auto-gen files

* don't render empty row if offers is empty

* change return val of ListAll to Offers

* fix build

* add method to check for empty offer when rendering template

* fix typo

* fix lint and typos

* lean out IsEmpty

* dont use named return vals

* better clarify offer statuses

* change back order of setting offer.Status

* lint

* satellite/marketingweb: allow disabling rewards (#2392)

* implement handler for stop offer endpoint

* use proper text and fix data-target for free-credit stop modal
2019-07-10 13:12:40 -04:00
Jess G
f9696d6c5e
satellite/metainfo: add buckets RPC and database (#2460)
* add db interface and methods, add sa metainfo endpoints and svc

* add bucket metainfo svc funcs

* add sadb bucekts

* bucket list gets all buckets

* filter buckets list on macaroon restrictions

* update pb cipher suite to be enum

* add conversion funcs

* updates per comments

* bucket settings should say default

* add direction to list buckets, add tests

* fix test bucket names

* lint err

* only support forward direction

* add comments

* minor refactoring

* make sure list up to limit

* update test

* update protolock file

* fix lint

* change per PR
2019-07-08 15:32:18 -07:00
Yingrong Zhao
e36f43c479
satellite/rewards: replace iota with hardcoded OfferType value (#2432)
* use 0 as the invalid offer type value

* add comment

* Update satellitedb.dbx.go

* Update satellitedb.dbx.go

* update comment
2019-07-08 14:39:56 -04:00
Jess G
0587dd79d6 change bucket table to fix conflict with pointerDB (#2452)
* change bucket table to fix conflict with pointerDB

* update init query

* update migration for dbx psql

* fix migration testdata
2019-07-04 00:03:56 +02:00
Jess G
0158b1eb5a
add bucket metadata table in SA masterDB (#2369)
* add bucket metadata table in SA masterDB

* fix indentation

* update db model per CR comments

* update testdata

* add missing field on sql testdata

* fix args to testdata

* unique bucket name

* fix fkey constraint for test

* fix one too many commas

*  update timestamp type

* Trigger Jenkins

* Trigger Jenkins yet again
2019-07-01 13:45:21 -07:00
Stefan Benten
01beaa289a
Mask IP Addresses to subnets (#2305) 2019-06-24 17:33:18 +02:00
Bill Thorp
8f47fca5d3
Remove audit / uptime ratio fields (#2247)
* removed ratios
2019-06-21 13:14:53 -04:00
Natalie Villasana
9386187fe6
add disqualification and new reputation system into overlay cache (#2227) 2019-06-20 09:56:04 -04:00
aligeti
7b450927ff
code updates with comments (#2245) 2019-06-19 08:50:50 -04:00
Egon Elbre
8f4a6afc8a Fixes for value attribution (#2238)
* Move value attribution to satellite/

* remove unnecessary conversion
2019-06-19 08:02:37 -04:00
Bill Thorp
81134e97cc
nodes db - adds alpha and beta reputation fields (#2215)
* node dbx and migrate adding alpha and beta reputation
2019-06-18 09:45:02 -04:00
aligeti
09940d4e0b
value attribution DB interface Insert & Get method support (#2200)
* value attribution DB interface methods support
2019-06-18 09:06:33 -04:00
Ivan Fraixedes
35c8648330
[v3-1914] Storage node disqualification: Change type from bool to timestamp (#2212)
* satellite/satellitedb: Alter nodes disqualification column
  Change the type of the 'disqualification' column of the nodes table from
  boolean to timestamp.
* overlay/cache: Change Disqualified field type
  Change the Disqualified field type the NodeDossier struct type from bool
  to time.Time to match with the disqualified type used by the DB layer.
* satellite/satellitedb: Update queries uses disqualified
  Update the queries which uses the disqualified column due to the column
  type has been changed from boolean to nullable timestamp.
* docs/design: Update disqualification due impl changes
  Update the disqualification design document to contain the architectural
  change required to be able to restore unfair disqualified nodes in case
  of an unexpected cause (bug, mistake, hard network disconnection, etc.).
2019-06-18 11:14:31 +02:00
Faris Huskovic
c0fe061a67 add user credits table (#2085)
* add user credits table

* change primary key, change type for credit_type, and change relation kind of foreign keys from cascade to restrict

* modify table and query methods

* modify schema

* add dbx queries

* add migration file

* add orderby to read available credit entries
2019-06-13 09:52:33 -04:00
ethanadams
380684e766 Removing bucket_id and adding project_id and bucket_name as primary key (#2182)
* removing bucket_id and adding project_id and bucket_name as primary key

* cleanup
2019-06-13 08:28:55 -04:00
Yaroslav Vorobiov
51db703b2b
Console add payments to service (#2100) 2019-06-06 19:07:14 +03:00
Egon Elbre
28a1201590 pkg/bwagreements: remove service (#2138) 2019-06-06 15:57:58 +02:00
Dennis Coyle
b9d586901e
Value Attribution DBX Model (#2116)
* adds model to satellite dbx

* cleans up model spacing

* generated golang from dbx

* added migration steps

* Added testdata

* changed node_id -> bucket_id

* adds -- NEW DATA -- to testdata

* more testdata changes

* adds -- NEW DATA -- line

* dbx makes the table plural

* missed a singular value_attribution

* restart jenkins

* Update satellitedb.dbx

* adjust to PR comments

* autogenerated dbx models

* restart jenkins
2019-06-05 12:06:14 -04:00
Yingrong Zhao
09b0c2a630
create db implementation for offer table (#2031)
* init marketing service

Fix linting error

Create offerdb implementation

Create offers service

Add update method

Create offer table and migration

Fix linting error

fix conflicts

Insert new data

Change duration to have clear indication to be based on days

add error wrapper

Change from using uuid to int for id field

* Create Marketing service

* make error virable name more readable

* add condition in update service method to check offer status

* generate lock file

Change get to listAllOffers

* Add method for getting current offer

wip

* add check for expires_at in update method

* Fix conflicts

* add copyright header

* Fix linting error

* only allow update to active offers

* add isDefault argument to GetCurrent

* Update lock file

* add migration file

* finish migrate for adding credit_in_cents for both award and invitee

* save 100 years as expiration date for default offers

* create crud test for offers

* add GetCurrent test

* modify doc

* Fix GetCurrent to work with default offer

* fix linting issue

* add more tests and address feedbacks

* fix migration file

* add type column back to match with mockup design

* add type column back to match with mockup design

* move doc changes to new pr

* add comments

* change GetCurrent to GetCurrentByType

* fix typo
2019-06-04 15:17:01 -04:00
Natalie Villasana
6db9388082 add disqualified column to nodes table (#2086)
* add disqualified column to nodes table, update migrate script and testdata

* fix crazy formatting of postgres.v25.sql
2019-05-30 17:38:23 -04:00
Michal Niewrzal
f731267e8c Per-project usage limiting (#2036)
What: Changes to support custom usage limit for the project. With this implementation by default project usage limit is taken from configuration flag. If project DB field usage_limit will be set to value larger than 0 it will become custom usage limit and we will be used to verify is limit was exceeded.

Whats changed:

usage_limit (bigint) field added to projects table (with migration)
things related to project usage moved from metainfo endpoint to project usage type
accounting.ProjectAccounting extended with GetProjectUsageLimits() method
Why: We need to have different usage limits per project. https://storjlabs.atlassian.net/browse/V3-1814
2019-05-28 09:36:52 -06:00
Jeff Wendling
1bd52b9f90 server side macaroons (#1945)
What: Adds macaroon support to the server side

Why: So that api keys are now macaroons
2019-05-24 10:51:27 -06:00
Yingrong Zhao
f9045f8385 Create offer table in satellitedb for free credit program (#1975) 2019-05-22 23:41:55 +02:00
Cameron
4058c29ca4
filter duplicate node IPs (#1890)
* add last_ip field to dbx model node, generate dbx

* add last_ip to node proto, generate pb

* migrate

* resolve address in transport.DialNode, update lastIp in cache.UpdateAddress

* use net.SplitHostPort to isolate host address from port

* define DistinctIPs flag

* add test for GetIP

* select last_ip when querying for nodes

* if distinctIPs flag == true, query for nodes with distinct IPs

* some basic tests

* change last_ip to field 14 in proto

* remove comments

* check err

* change distinctIPs to distinctIP

* exclude IPs from newNodes in query for reputable nodes

* add index on last_ip

* only add to excludedIPs if flag is true

* test half new nodes returns distinct IPs

* fix alignment

* add test

* rework ip filter query, add retry logic, add switch for database driver

* add retry to SelectNewNodes

* change discovery intervals so IPs don't get overwritten

* remove TestGetIP

* edit updating node stats in test

* split exclude into nodeIDs and IPs

* separate non-distinct IP query into other function

* trigger checks

* remove else block
2019-05-22 16:06:27 -04:00
Maximillian von Briesen
cc020dfdea
Create containment mode database table and migrate scripts (#1970) 2019-05-16 10:11:15 -04:00
Bogdan Artemenko
f46487b015
ResetPassword Table and all CRUD methods. (#1916) 2019-05-13 18:53:52 +03:00
Jennifer Li Johnson
5395ff5fe6
Refactor accountingdb interface (#1897)
* splits accounting db into storagenodeaccounting and projectaccounting interfaces and renames methods to match
2019-05-10 15:05:42 -04:00
Egon Elbre
ecde1bd251 jenkins: use -race for check-imports and fix dbx check (#1873)
What: Use -race for check-imports, this means it will use the cached build files.

Why:
2019-05-01 09:44:12 -06:00
Egon Elbre
fe2bf7d4a6 fix saving uplink public key on satellite (#1797)
* fix save into cert records

* fix transaction logic

* test shouldn't be flaky anymore

* add missing cause method

* restart jenkins
2019-04-23 15:48:57 -04:00
Egon Elbre
f7ed63a119
handle database error checks properly (#1796) 2019-04-23 14:13:57 +03:00
Bill Thorp
17a227e6e9
refactor injuredsegments db so that we can't have duplicates (#1717)
made repairqueue not use a true queue, forbid duplicates
2019-04-16 14:14:09 -04:00
Stefan Benten
bae4c820ee
Add Version Information into KAD Network and SatelliteDB & Change Selection Process (#1648)
* Initial Webserver Draft for Version Controlling

* Rename type to avoid confusion

* Move Function Calls into Version Package

* Fix Linting and Language Typos

* Fix Linting and Spelling Mistakes

* Include Copyright

* Include Copyright

* Adjust Version-Control Server to return list of Versions

* Linting

* Improve Request Handling and Readability

* Add Configuration File Option
Add Systemd Service file

* Add Logging to File

* Smaller Changes

* Add Semantic Versioning and refuses outdated Software from Startup (#1612)

* implements internal Semantic Version library

* adds version logging + reporting to process

* Advance SemVer struct for easier handling

* Add Accepted Version Store

* Fix Function

* Restructure

* Type Conversion

* Handle Version String properly

* Add Note about array index

* Set temporary Default Version

* Add Copyright

* Adding Version to Dashboard

* Adding Version Info Log

* Renaming and adding CheckerProcess

* Iteration Sync

* Iteration V2

* linting

* made LogAndReportVersion a go routine

* Refactor to Go Routine

* Add Context to Go Routine and allow Operation if Lookup to Control Server fails

* Handle Unmarshal properly

* Linting

* Relocate Version Checks

* Relocating Version Check and specified default Version for now

* Linting Error Prevention

* Refuse Startup on outdated Version

* Add Startup Check Function

* Straighten Logging

* Dont force Shutdown if --dev flag is set

* Create full Service/Peer Structure for ControlServer

* Linting

* Straighting Naming

* Finish VersionControl Service Layout

* Improve Error Handling

* Change Listening Address

* Move Checker Function

* Remove VersionControl Peer

* Linting

* Linting

* Create VersionClient Service

* Renaming

* Add Version Client to Peer Definitions

* Linting and Renaming

* Linting

* Remove Transport Checks for now

* Move to Client Side Flag

* Remove check

* Linting

* Transport Client Version Intro

* Adding Version Client to Transport Client

* Add missing parameter

* Adding Version Check, to set Allowed = true

* Set Default to true, testing

* Restructuring Code

* Uplink Changes

* Add more proper Defaults

* Renaming of Version struct

* Dont pass Service use Pointer

* Set Defaults for Versioning Checks

* Put HTTP Server in go routine

* Add Versioncontrol to Storj-Sim

* Testplanet Fixes

* Linting

* Add Error Handling and new Server Struct

* Move Lock slightly

* Reduce Race Potentials

* Remove unnecessary files

* Linting

* Add Proper Transport Handling

* small fixes

* add fence for allowed check

* Add Startup Version Check and Service Naming

* make errormessage private

* Add Comments about VersionedClient

* Linting

* Remove Checks that refuse outgoing connections

* Remove release cmd

* Add Release Script

* Linting

* Update to use correct Values

* Change Timestamp handling

* Adding Protobuf changes back in

* Adding SatelliteDB Changes and adding Storj Node Version to PB

* Add Migration Table

* Add Default Stats for Creation

* Move to BigInt

* Proper SQL Migration

* Ensure minimum Version is passed to the node selection

* Linting...

* Remove VersionedClient and adjust smaller changes from prior merge

* Linting

* Fix PB Message Handling and Query for Node Selection

* some future-proofing type changes

Change-Id: I3cb5018dcccdbc9739fe004d859065992720caaf

* fix a compiler error

Change-Id: If66bb92d8b98e31cd618ecec9c6448ab9b037fa5

* Comment on Constant for Overlay

* Remove NOT NULL and add epoch call as function

* add versions to bootstrap and satellites

Change-Id: I436944589ea5f21600cdd997742a84fe0b16e47b

* Change Update Migration

* Fix DB Migration

* Increase Timeout temporarily, to see whats going on

* Remove unnecessary const and vars
Cleanup Function calls from deprecated NodeVersion struct

* Updated Protopuf, removed depcreated Code from Inspector

* Implement NodeVersion into InfoResponse

* Regenerated locked.go

* Linting

* Fix Tests

* Remove unnecessary constant

* Update Function and Flag Description

* Remove Empty Stat Creation

* return properly with error

* Remove unnecessary struct

* simplify migration step

* Update Inspector to return Version Info

* Update local Endpoint Version Handling

* Reset Travis Timeout

* Add Default for CommitHash

* single quotes
2019-04-10 08:04:24 +02:00
Natalie Villasana
2533aa1ab1
logs dbx constraint violation error (#1712) 2019-04-08 18:03:48 -04:00
Jennifer Li Johnson
8549421385
Remove bw from tally service + query bandwidth in rollup service (#1618) 2019-04-04 11:20:59 -04:00