Go to file
paul cannon c053bdbd70 satellite/satellitedb: prepare to remove big.Float from db
Why: big.Float is not an ideal type for dealing with monetary amounts,
because no matter how high the precision, some non-integer decimal
values can not be represented exactly in base-2 floating point. Also,
storing gob-encoded big.Float values in the database makes it very hard
to use those values in meaningful queries, making it difficult to do
any sort of analysis on billing.

Now that we have amounts represented using monetary.Amount, we can
simply store them in the database using integers (as given by the
.BaseUnits() method on monetary.Amount).

We should move toward storing the currency along with any monetary
amount, wherever we are storing amounts, because satellites might want
to deal with currencies other than STORJ and USD. Even better, it
becomes much clearer what currency each monetary value is _supposed_ to
be in (I had to dig through code to find that out for our current
monetary columns).

Deployment
----------

Getting rid of the big.Float columns will take multiple deployment
steps. There does not seem to be any way to make the change in a way
that lets existing queries continue to work on CockroachDB (it could be
done with rules and triggers and a stored procedure that knows how to
gob-decode big.Float objects, but CockroachDB doesn't have rules _or_
triggers _or_ stored procedures). Instead, in this first step, we make
no changes to the database schema, but add code that knows how to deal
with the planned changes to the schema when they are made in a future
"step 2" deployment. All functions that deal with the
coinbase_transactions table have been taught to recognize the "undefined
column" error, and when it is seen, to call a separate "transition shim"
function to accomplish the task. Once all the services are running this
code, and the step 2 deployment makes breaking changes to the schema,
any services that are still running and connected to the database will
keep working correctly because of the fallback code included here. The
step 2 deployment can be made without these transition shims included,
because it will apply the database schema changes before any of its code
runs.

Step 1:

    No schema changes; just include code that recognizes the
    "undefined column" error when dealing with the
    coinbase_transactions or stripecoinpayments_tx_conversion_rates
    tables, and if found, assumes that the column changes from Step
    2 have already been made.

Step 2:

    In coinbase_transactions:

     * change the names of the 'amount' and 'received' columns to
       'amount_gob' and 'received_gob' respectively
     * add new 'amount_numeric' and 'received_numeric' columns with
       INT8 type.

    In stripecoinpayments_tx_conversion_rates:

     * change the name of the 'rate' column to 'rate_gob'
     * add new 'rate_numeric' column with NUMERIC(8, 8) type

    Code reading from either of these tables must query both the X_gob
    and X_numeric columns. If X_numeric is not null, its value should
    be used; otherwise, the gob-encoded big.Float in X_gob should be
    used. A chore might be included in this step that transitions values
    from X_gob to X_numeric a few rows at a time.

Step 3:

    Once all prod satellites have no values left in the _gob columns, we
    can drop those columns and add NOT NULL constraints to the _numeric
    columns.

Change-Id: Id6db304b404e6fde44f5a8c23cdaeeaaa2324f20
2021-09-29 00:23:44 +00:00
.github .github: add issue templates 2021-09-24 08:42:48 +00:00
certificate all: fix mon.Task leak 2021-08-05 14:07:45 +03:00
cmd satellite/repair: update audit records during repair 2021-09-24 00:48:13 +00:00
crashcollect crashcollect: removed redundant structure 2021-04-28 00:35:39 +03:00
docs docs: move contribution guide to repository root 2021-09-28 07:38:11 +00:00
installer/windows storj/storj: more domain changes 2021-04-15 20:51:43 +00:00
multinode munltinode/console: node online statuses extended with StatusUnauthorized and StatusStorageNodeInternal 2021-07-21 15:00:41 +03:00
private satellite/repair: update audit records during repair 2021-09-24 00:48:13 +00:00
resources cmd: add ca-certificates to Docker images (#3986) 2020-12-08 01:38:33 +01:00
satellite satellite/satellitedb: prepare to remove big.Float from db 2021-09-29 00:23:44 +00:00
scripts ci: switch back to pulling gateway@main 2021-09-28 15:01:04 +00:00
storage storage/filestore: test interaction between VerifyDirReadable and VerifyDirWriteable 2021-09-15 18:57:56 +03:00
storagenode storagenode/piecestore: simplify TestTooManyRequests 2021-09-21 17:18:37 +03:00
testsuite satellite/metabase/segmentloop: start immediately on manual trigger 2021-09-28 18:32:03 +00:00
versioncontrol ci: fix lint errors 2021-05-17 13:37:31 +00:00
web web/satellite, testsuite/satellite/ui: removed onb CLI flow's Generate AG step and updated tests 2021-09-24 13:21:40 +00:00
.dockerignore Forward-port release-alpha8 build script issues (#1726) 2019-04-09 23:01:10 -06:00
.gitignore satellite/console/wasm: reduce size to <9MB 2020-12-14 16:41:39 +00:00
CODE_OF_CONDUCT.md Adding CODE_OF_CONDUCT to storj/storj repo (#779) 2018-12-07 15:10:02 -05:00
CONTRIBUTING.md docs: move contribution guide to repository root 2021-09-28 07:38:11 +00:00
go.mod satellite/metabase/segmentloop: start immediately on manual trigger 2021-09-28 18:32:03 +00:00
go.sum satellite/metabase/segmentloop: start immediately on manual trigger 2021-09-28 18:32:03 +00:00
Jenkinsfile rollingupgrade: add migration to metabase 2021-03-25 10:52:35 +00:00
Jenkinsfile.public ci: switch back to pulling gateway@main 2021-09-28 15:01:04 +00:00
Jenkinsfile.ui jenkinsfile: bump build timeout 2021-09-27 18:31:16 +00:00
LICENSE license code with agplv3 (#126) 2018-07-05 10:24:26 -04:00
MAINTAINERS.md Create MAINTAINERS.md 2021-07-07 13:27:07 +02:00
Makefile ci: switch back to pulling gateway@main 2021-09-28 15:01:04 +00:00
monkit.lock satellite/audit: log error and increment metric if shares cannot be verified 2021-08-27 15:28:16 +00:00
package-lock.json Satellite api keys frontend (#1039) 2019-02-01 18:19:30 +02:00
proto.lock multinode/payouts: paystub extended with disposed 2021-06-01 16:28:20 +00:00
README.md docs: move contribution guide to repository root 2021-09-28 07:38:11 +00:00

Storj V3 Network

Go Report Card Go Doc Coverage Status Alpha

Storj is building a decentralized cloud storage network. Check out our white paper for more info!


Storj is an S3-compatible platform and suite of decentralized applications that allows you to store data in a secure and decentralized manner. Your files are encrypted, broken into little pieces and stored in a global decentralized network of computers. Luckily, we also support allowing you (and only you) to retrieve those files!

Table of Contents

Contributing to Storj

All of our code for Storj v3 is open source. If anything feels off, or if you feel that some functionality is missing, please check out the contributing page. There you will find instructions for sharing your feedback, building the tool locally, and submitting pull requests to the project.

A Note about Versioning

While we are practicing semantic versioning for our client libraries such as uplink, we are not practicing semantic versioning in this repo, as we do not intend for it to be used via Go modules. We may have backwards-incompatible changes between minor and patch releases in this repo.

Start using Storj

Our wiki has documentation and tutorials. Check out these three tutorials:

License

The network under construction (this repo) is currently licensed with the AGPLv3 license. Once the network reaches beta phase, we will be licensing all client-side code via the Apache v2 license.

For code released under the AGPLv3, we request that contributors sign our Contributor License Agreement (CLA) so that we can relicense the code under Apache v2, or other licenses in the future.

Support

If you have any questions or suggestions please reach out to us on our community forum or file a ticket at https://support.storj.io/.