This change adds two new tables to process orders as fast as we used
to but in an asynchronous manner and with hopefully less storage
usage. This should help scale on cockroach, but limits us to one
worker. It lays the groundwork for the order processing pipeline to
be queue rather than database driven.
For more details, see the added fast billing changes blueprint.
It also fixes the orders db so that all the timestamps that are
passed to columns that do not contain a time zone are converted to
UTC at the last possible opportunity, making it less likely to use
the APIs incorrectly. We really should migrate to include timezones
on all of our timestamp columns.
Change-Id: Ibfda8e7a3d5972b7798fb61b31ff56419c64ea35
This document describes a new state, suspension, which nodes will
be placed in if they respond to too many audits with "unknown" errors.
When in this state, node operators will have to fix their node within a
certain amount of time, or they will be disqualified. While nodes are
suspended, they will not be able to receive new data.
Change-Id: I543f27ada66d8b40fc52ca8ead8ecc8479326bad
- also updated ping chore to pick up trust changes
- fixed small typo in blueprint
- fixed flags for storj-sim
- wired up changes to testplanet
Change-Id: I02982f3a63a1b4150b82a009ee126b25ed51917d
* change satellite.Peer name to Core
* change to Core in testplanet
* missed a few places
* keep shared stuff in peer.go to stay consistent with storj/docs
* archive old referral offer doc
* init referral manager
* add Referral Manager CLI definition
* add draft implementation steps
* use existing registration_token table for storing invitation tokens
* fix numbering
* add open question about auth
* wip pseudocode
* add user interface design
* separate token generation step from getting users step
* change button to be a referral tab on the UI
* simplify some sentences. clarified CLI referral manager interaction
* add prevent two processes running at the same time
* add cli code
* clear users when CLI exites earlier or referral manager finishes sending tokens to satellites
* use transaction for saving tokens on satellite so we can roll back if error occurs
* add auth token and GetSatelliteURLs
* add schema for satellites table
* update to only have satelite to talk to referral manager
* add definition for eligible users and owner's satellite
* fix indentation
* add saving user info during redeem pseudocode
* wip
* address reviews
* add unredeemed token in users table
* address comments
we've decided to rename design docs to blueprints to indicate that we don't intend to keep blueprints up to date once completed. instead, the final step of blueprint implementation is to archive the blueprint and update actual documentation elsewhere (potentially copying and pasting most of the blueprint). this PR updates the template and readme
Bluepints are replacing the design documents.
A blueprint has a wrapup section, this commit adds the section to the
document and replace any mention to design document by blueprint.
* add md extension to dd process doc
* rm old format of dd process doc
* wip design doc
* add diagram, add implementation
* adjust format
* fix format
* add updates per CR and arch review meeting
* update diagram sizes
* make image smaller
* try using svg instead of png
* replace png with svg, mv to docs/images dir
* updates per CR comments
* more CR comment changes
* update sa design img
* grammar fixes
* a few more updates
* fixings nits
* fix spelling err
* fix spelling, change binary to process, rm process doc
* add changes to design imagees per CR comments
This way the design doc process is directly visible when browsing the doc/design folder in the GitHub Web UI. As the number of docs grows it would be harder to find the doc with the design process.
* design doc: path component encoding
in talks with a number of people on the team, it seems there is some
opportunity for coordination on how paths should be encoded so that
everyone works works well with each other. in particular, we care
about matching s3 semantics for list for unencrypted paths which
contain empty path components. that makes it hard to increment
and decrement keys for seek operations. additionally, there is
work to reduce our metadata overhead for storage of paths. this
proposal takes most of the ideas from that work and ensures that
it supports the ability to efficiently seek through prefixes of
paths.
* fix following and discuss c strings
* added satalite partner value attribution report. WIP
* WIP
* basic attribution report test completed. still a WIP
* cleanup
* fixed projectID conversion
* report display cleanup
* cleanup .added more test data
* added partnerID to query results
* fixed lint issues
* fix import order
* suggestions from PR review
* updated doc to reflect implementation
* clarification comments in the report SQL
* Changed based on PR suggestion
* More changes based on PR suggestions
* Changes based on PR suggestions
* reordered tests to make consistant with previous 2
* small comments cleanup
* More PR suggestions
* fixed lint issue and removed printf
* fixed var name
* Updates based on PR suggestions
* fixed message
* fixed test
* changes required after merge from master
* initial design doc for SN graceful exit
* updates to the document mostly syntax and typos
* updates based on the first acceptance meeting
* SN will get paid for download data from clients
* updates based on our second acceptance meeting
* typo updates
* updates to things and stuff based on comments
* updated based on comment
* syntax changes/ typo fix
* satellite/satellitedb: Alter nodes disqualification column
Change the type of the 'disqualification' column of the nodes table from
boolean to timestamp.
* overlay/cache: Change Disqualified field type
Change the Disqualified field type the NodeDossier struct type from bool
to time.Time to match with the disqualified type used by the DB layer.
* satellite/satellitedb: Update queries uses disqualified
Update the queries which uses the disqualified column due to the column
type has been changed from boolean to nullable timestamp.
* docs/design: Update disqualification due impl changes
Update the disqualification design document to contain the architectural
change required to be able to restore unfair disqualified nodes in case
of an unexpected cause (bug, mistake, hard network disconnection, etc.).
* change database schema based on new user flow
* Add offer db implementation
* Change credit duration unit to all be days
* revert back to use cent to be the unit for credit
* Add comments for schema
* add type back for mockup design
* initial password key derivation design
* some typos/clarifications
* point out that clients can ignore bucket entropy
* clarification about offline table generation
* change proposal to only do network requests when inputting passwords
* make algorithm have shorter lines
* fix algorithm example
* be explicit about allowing already-formed root keys
* go back to having a salt in the project/bucket metadata
* some proofreading/clarifications
* feedback and clarifications
* clarify definition of root key
* add uplink-contexts design doc
* respond to some feedback and rename context to scope
* feedback summaries
* review fixes
* review feedback
* more review feedback
* fix types in go structs
* init doc
* Add link to offer types table
* Correct title
* Update Credits interface
* Add another question
* Add steps for configure segment.io for different satellite
* Update database design
Change database schema
* Address comments
* Add explaination on how to restrict access to the private port
* rename file
* Add credits_used column and offer update limit
* Change credits to be integer
* Remove extra blank lines
* Initial version
* small update
* add probability of false positive table
* put the table at the right place
* updates during meeting
* adds some updates about using bloom filters for gc deleted data
* small fixes
* adds implementation steps and basic gc service protobuf
* adds super brief abstract
* rm gendered language
* removes DataDeletionReport section, adds rationale for not using 2 bloom filters
* Remove parts about deletion
* rearrangement
