Commit Graph

53 Commits

Author SHA1 Message Date
Jeremy Wharton
e34826c168 docs/blueprints: add automatic account freezing/unfreezing blueprint
This blueprint describes a way to allow user accounts to be
automatically restricted after failing to pay and unrestricted after
payment has been collected.

Change-Id: I4b64eb8ef6bf83603be7ad7d7bcb78f2d885c2a4
2022-11-03 16:56:28 +00:00
paul cannon
0c3dd44490 docs: audit-scaling: clarify process structure
and clarify some related implementation details.

Most notably, this change clarifies that the verification audit workers
and reverification audit workers are meant to live in a process or
processes separate from the satellite core, and outlines an extra queue
that will be used for communication with the core.

It's not entirely clear to me that this is the right approach; we would
save some fairly significant implementation time by leaving both types
of worker in the core. That would make it necessary to reconfigure and
restart the core when we wanted to change the number of verification
and/or reverification workers, and scaling would be limited to the
computational capacity of the core vm, but maybe those are acceptable
conditions.

Another option would be to leave the Verifier workers in the core, and
having a separate process for Reverifiers. That would be sort of a
middle way between the two above.

Change-Id: Ida12e423b94ef6088733b13d5cc58bdb78f2e93f
2022-10-27 20:00:36 +00:00
Erik van Velzen
0cfbd34da2 docs/blueprints: byte range multipart copy
Design proposal for approval to support S3's UploadPartCopy. Major
revision 2.

Change-Id: I9cbe934a773d3496101f32afb19492ea75f48061
2022-10-13 12:03:17 +00:00
Jennifer Johnson
182b94aab2 docs/blueprints: design doc for changes needed to allow for scaling audit workers
Change-Id: I3e214c900a02ba15c2fb47558f0bc75310db2a7c
2022-09-28 19:47:41 +00:00
Egon Elbre
0513d4cf3a docs/blueprints: add async packing
Describe a process for packing small files.

Change-Id: I9f6ad9a85ccf0a92705a47e2ff8efa0072e65230
2022-04-23 06:34:17 +00:00
Moby von Briesen
3397886b14 docs/blueprints: Add session management blueprint
New blueprint describes a design which provides the satellite greater
control over sessions authorized to use the web app.

Change-Id: I5af227aef6d6b096167e2e8a60f1e8214c2cd71f
2022-04-06 21:10:33 +00:00
Fadila Khadar
62eeb599ce docs/blueprints: server-side copy
https://github.com/storj/team-metainfo/issues/23

Change-Id: Ie4b7262cac6191aa1fe72b65cbaaa6f89d590a05
2022-02-16 17:26:23 +00:00
Jeremy Wharton
c968bd0b73 docs/blueprints: Create API renovation blueprint
This change adds a blueprint outlining the renovation of the
current API system including guidelines for API code generation.

Change-Id: I5560cc8642eee44842829ecddf205886cd5b13f2
2022-01-24 14:24:31 +00:00
Elek, Márton
debb82f596
Geofencing and advanced placement-constraint support (#4227) 2021-11-15 15:23:41 +02:00
Maximillian von Briesen
036687b58a
docs/blueprints: Create coupon codes blueprint (#4024)
Change-Id: If7b364d1c6db389feaff275d95acf8d9c8e08c81
2021-06-21 19:24:01 -04:00
Michał Niewrzał
60ff87a7d7 design doc: server-side move
Change-Id: Ic6ae180c7a2288bbf463adcf4ca371b9a9222ae1
2021-05-07 07:48:48 +00:00
JT Olio
35d8a840d3 storj/storj: more domain changes
Change-Id: I643c38bdae6dc26c9346147d80a83ae9dde2eeae
2021-04-15 20:51:43 +00:00
JT Olio
932039ebdf blueprint: layer2 support for zksync
Change-Id: I130ce4c6d04b6f56acdbaefff507c694e8bf8260
2021-03-01 23:41:11 +00:00
Cameron Ayer
ca12e98d5d docs: change 'offline score' to 'online score' in downtime tracking doc
Change doc to reflect a change made during the implementation.

Change-Id: I31f6a2c08ae97cba7068975ab6c9dd276726273a
2020-12-29 15:31:20 +00:00
Stefan Benten
7f1871b8f1 all: switch from master to main 2020-12-28 22:59:06 +01:00
Jeff Wendling
876e1be3b5 Blueprint: Sparse Order Storage
Change-Id: I1cf6f3bda84c9b9d5ccfbbaf51813d6ea9a65679
2020-12-21 18:04:54 +00:00
nerdatwork
5c34b62bde
Fixed typos in downtime tracking with audits doc (#3977)
Co-authored-by: Ivan Fraixedes <ivan@fraixed.es>
2020-11-27 17:25:21 +01:00
Hector Fernandez
dc5a5df7f5
chore: fix typos in the documentation (#3959) 2020-11-09 22:00:34 +02:00
Yehor Butko
c6415406a1
docs/blueprints: graceful exit initial refactoring (#3938)
* docs: update graceful exit refactoring doc

Co-authored-by: paul cannon <thepaul@storj.io>
Co-authored-by: Jennifer Li Johnson <jennifer@storj.io>
Co-authored-by: Maximillian von Briesen <mobyvb@gmail.com>
2020-10-19 23:34:48 -05:00
Ivan Fraixedes
0525858b72 docs/blueprints: Fix typos horizontal scale GC
Fix a few typos in the horizontal scale garbages collection blueprint
document.

Change-Id: I25a0e59ae526e6f270ea4e8b5eac36d779d597dc
2020-08-11 13:53:52 +00:00
Jessica Grebenschikov
f27285197d docs/blueprint: scale garbage collection
Change-Id: Iacd6f7f0a24ce91fa3da21a04707f3a6d27766f6
2020-08-03 23:52:40 +00:00
JT Olio
a974db7eec Update automatic updater blueprint to catch missing requirement
from https://github.com/storj/storj/pull/2677#pullrequestreview-270882629

this bit evidently never made it in.

See:
  https://review.dev.storj.io/c/storj/private/+/2053
  https://review.dev.storj.io/c/storj/storj/+/2054

Change-Id: Ie1435867cefa6d7e8b9f71bca13e24a161a619ae
2020-06-05 16:24:23 -06:00
paul cannon
5ed1985b22 docs/blueprints: trusted-delegated-repair.md
Trusted Delegated Repair is performing repair on trusted servers outside
the network boundary of the satellite.

Change-Id: I13b0e6f53100cefcb89c0c77bd1421c549551ab7
2020-06-04 18:22:25 +00:00
Isaac Hess
df0ef7e0cd docs: update access-revocation blueprint
This update includes the plan for authorizing macaroon revocation
requests.

Change-Id: I859fa7c23f3bc1c8a92741df6f88187547613303
2020-06-01 16:33:13 +00:00
Fadila
36286a0629
docs/blueprints: installation and auto-update on Linux (#3463) 2020-06-01 18:05:24 +02:00
Cameron
d5c86aa987
docs/blueprints: tracking downtime with audits (#3834)
* docs/blueprints: add blueprint for downtime DQ
2020-05-29 11:23:32 -04:00
Isaac Hess
5364a9f74d docs/blueprints: Add access-revocation.md
Change-Id: I74d9cc5564c727430b49d6b6f9040fd7df6f54e6
2020-05-26 18:59:27 +00:00
Egon Elbre
ec589a8289 all: fix comments about grpc
Change-Id: Id830fbe2d44f083c88765561b6c07c5689afe5bd
2020-05-11 13:05:34 +03:00
Jess G
73b2cd3fd5
docs/blueprints: blueprint nodes cache (#3838)
* init design for selecting nodes for uploads

Change-Id: I40920a88fc8c0c8591eaf3f4e711578ca02dad73

* spelling erro

Change-Id: I843945f356dbdfd5c8b7cefee91ab4172c41f3fb

* updates per comments

Change-Id: I2d6f0f9b4f22c4c7ae8ed80c4f533d2d9392badf

* add details

Change-Id: Id31ecfbe9f7ff068b127f04ba061a3fb9ad254cc

* mv redis to rationale section

Change-Id: I98d8565bb228555d92682d3b08c99c6056b0a84b

* add details about ReliabilityCache

Change-Id: I9f0ecf0a96a217c5e55c6a855b8e06e0a76ab574

* add option details

Change-Id: Icd229cbbdc263466d16856fdd82a021509a1c76a

* add opt for queue

Change-Id: I9d36e5e6f923440b6c22066a2aaf981aecc584d6

* final updates

Change-Id: Iee74985f3a45b3ef58ff23ef3ae26f609fb7f0d5
2020-05-06 07:44:21 -07:00
Yaroslav Vorobiov
2b00d78add satellite/payments: add documentation
Change-Id: I75712ef282530b56ea7ca1f2bc31ee0af43dae33
2020-04-30 21:12:16 +00:00
Caleb Case
921b574554
docs/blueprints: slow down and retry (#3826) 2020-04-15 00:25:08 +02:00
JT Olio
67a19ec958 docs/blueprints: update link to uplink docs
Change-Id: I178d6518d2f7f26ac20b7e1efa7c52a5daa8748d
2020-04-13 21:00:31 -06:00
Egon Elbre
e8f18a2cfe private/testplanet: expose storagenode and satellite Config
Change-Id: I80fe7ed8ef7356948879afcc6ecb984c5d1a6b9d
2020-03-27 17:01:25 +02:00
Simon Guindon
594d6e03aa docs/blueprints: Add design doc for distributed tracing.
Change-Id: I98f76f857d1a6ccd384adc6287137b46e37b9904
2020-02-25 20:29:05 +00:00
Isaac Hess
e486a073cb docs: Add uplink telemetry doc
Change-Id: I6f47ef4af80d0c76a32dc360f8809a526a4e948f
2020-02-25 17:52:34 +00:00
Jeff Wendling
f671eb2beb satellite/satellitedb: use queue for orders to get back fast billing
This change adds two new tables to process orders as fast as we used
to but in an asynchronous manner and with hopefully less storage
usage. This should help scale on cockroach, but limits us to one
worker. It lays the groundwork for the order processing pipeline to
be queue rather than database driven.

For more details, see the added fast billing changes blueprint.

It also fixes the orders db so that all the timestamps that are
passed to columns that do not contain a time zone are converted to
UTC at the last possible opportunity, making it less likely to use
the APIs incorrectly. We really should migrate to include timezones
on all of our timestamp columns.

Change-Id: Ibfda8e7a3d5972b7798fb61b31ff56419c64ea35
2020-02-24 17:07:07 +00:00
Maximillian von Briesen
a5afa4834a
docs/blueprints: Add blueprint for creating a new state for storagenodes, suspension (#3778)
This document describes a new state, suspension, which nodes will
be placed in if they respond to too many audits with "unknown" errors.
When in this state, node operators will have to fix their node within a
certain amount of time, or they will be disqualified. While nodes are
suspended, they will not be able to receive new data.

Change-Id: I543f27ada66d8b40fc52ca8ead8ecc8479326bad
2020-02-19 14:29:48 -05:00
Andrew Harding
cb89496569 storagenode/trust: wire up list into pool
- also updated ping chore to pick up trust changes
- fixed small typo in blueprint
- fixed flags for storj-sim
- wired up changes to testplanet

Change-Id: I02982f3a63a1b4150b82a009ee126b25ed51917d
2019-12-13 20:32:50 +00:00
Ivan Fraixedes
46ad27c5f2
docs/blueprints: Change adapt endpoint by creating (#3601)
Replace a sentence which indicates that we're creating and new endpoint
in the storage node side rather than adapting the current one.
2019-11-19 14:28:19 +01:00
Ivan Fraixedes
b5b4543df3
docs/blueprints: Deletion performance (#3520)
Creates the blueprint document for changing the mechanism to delete pieces and improve its performance.
2019-11-19 08:16:35 +01:00
Michal Niewrzal
89efd17f4d
docs/design: zombie segments cleaner (#3461) 2019-11-12 00:25:51 -08:00
Andrew Harding
0885eaa183 Storage Node Satellite Selection design (#3353) 2019-11-05 18:07:08 -05:00
Jess G
5abb91afcf
satellite: change the Peer name to Core (#3472)
* change satellite.Peer name to Core

* change to Core in testplanet

* missed a few places

* keep shared stuff in peer.go to stay consistent with storj/docs
2019-11-04 11:01:02 -08:00
Maximillian von Briesen
472672d52f
docs/storage-node-downtime-tracking: add open issue regarding satellite outages (#3370) 2019-10-25 11:39:11 -04:00
Brandon Iglesias
84156d6912 docs/blueprints: updating term to held amount (#3248) 2019-10-11 21:05:50 +03:00
Ethan Adams
4c4519f0be
satellite/gracefulexit: add transfer queue for pieces (#3174)
initial impl of transfer queue
updated docs represent the new design how we handle durability during exit
2019-10-07 16:38:05 -04:00
Yingrong Zhao
d29946d3d7
docs/blueprints: referral manager v1 (#3038)
* archive old referral offer doc

* init referral manager

* add Referral Manager CLI definition

* add draft implementation steps

* use existing registration_token table for storing invitation tokens

* fix numbering

* add open question about auth

* wip pseudocode

* add user interface design

* separate token generation step from getting users step

* change button to be a referral tab on the UI

* simplify some sentences. clarified CLI referral manager interaction

* add prevent two processes running at the same time

* add cli code

* clear users when CLI exites earlier or referral manager finishes sending tokens to satellites

* use transaction for saving tokens on satellite so we can roll back if error occurs

* add auth token and GetSatelliteURLs

* add schema for satellites table

* update to only have satelite to talk to referral manager

* add definition for eligible users and owner's satellite

* fix indentation

* add saving user info during redeem pseudocode

* wip

* address reviews

* add unredeemed token in users table

* address comments
2019-10-07 09:38:05 -04:00
Ethan Adams
9edfb6efe0
satellite/satellitedb: Initial GE Satellite DB Implementation (#3049)
Initial GE Satellite DB impl
Add basic CRUD operations for graceful_exit_progress and graceful_exit_transfer_queue tables.
2019-09-25 11:12:44 -06:00
Ethan Adams
886041e0ba
satellite/satellitedb: add new graceful exit tables and add graceful exit fields to nodes (#3033)
DB schema changes for satellite to support Graceful Exit
2019-09-13 12:57:32 -04:00
Yingrong Zhao
9f2f1527c5
storagenode/storagenodedb: add new tables for graceful exit (#3008)
* add database schema

* add migration

* change table name and update blueprint
2019-09-11 18:57:53 -04:00