Commit Graph

1336 Commits

Author SHA1 Message Date
Maximillian von Briesen
4d925f783c
Fix repairer unit test (#1557) 2019-04-03 19:00:25 -04:00
Dennis Coyle
88f0d63f33 Adds the AMD64 and ARM Storagenode Dockerfiles and setup scripts (#1544)
* add alpha dockerfile structure

* adds Stefans satelitte to white list
2019-04-03 14:17:55 -06:00
Stefan Benten
2cf86703a3
Add Versioning Server (#1576)
* Initial Webserver Draft for Version Controlling

* Rename type to avoid confusion

* Move Function Calls into Version Package

* Fix Linting and Language Typos

* Fix Linting and Spelling Mistakes

* Include Copyright

* Include Copyright

* Adjust Version-Control Server to return list of Versions

* Linting

* Improve Request Handling and Readability

* Add Configuration File Option
Add Systemd Service file

* Add Logging to File

* Smaller Changes

* Add Semantic Versioning and refuses outdated Software from Startup (#1612)

* implements internal Semantic Version library

* adds version logging + reporting to process

* Advance SemVer struct for easier handling

* Add Accepted Version Store

* Fix Function

* Restructure

* Type Conversion

* Handle Version String properly

* Add Note about array index

* Set temporary Default Version

* Add Copyright

* Adding Version to Dashboard

* Adding Version Info Log

* Renaming and adding CheckerProcess

* Iteration Sync

* Iteration V2

* linting

* made LogAndReportVersion a go routine

* Refactor to Go Routine

* Add Context to Go Routine and allow Operation if Lookup to Control Server fails

* Handle Unmarshal properly

* Linting

* Relocate Version Checks

* Relocating Version Check and specified default Version for now

* Linting Error Prevention

* Refuse Startup on outdated Version

* Add Startup Check Function

* Straighten Logging

* Dont force Shutdown if --dev flag is set

* Create full Service/Peer Structure for ControlServer

* Linting

* Straighting Naming

* Finish VersionControl Service Layout

* Improve Error Handling

* Change Listening Address

* Move Checker Function

* Remove VersionControl Peer

* Linting

* Linting

* Create VersionClient Service

* Renaming

* Add Version Client to Peer Definitions

* Linting and Renaming

* Linting

* Remove Transport Checks for now

* Move to Client Side Flag

* Remove check

* Linting

* Transport Client Version Intro

* Adding Version Client to Transport Client

* Add missing parameter

* Adding Version Check, to set Allowed = true

* Set Default to true, testing

* Restructuring Code

* Uplink Changes

* Add more proper Defaults

* Renaming of Version struct

* Dont pass Service use Pointer

* Set Defaults for Versioning Checks

* Put HTTP Server in go routine

* Add Versioncontrol to Storj-Sim

* Testplanet Fixes

* Linting

* Add Error Handling and new Server Struct

* Move Lock slightly

* Reduce Race Potentials

* Remove unnecessary files

* Linting

* Add Proper Transport Handling

* small fixes

* add fence for allowed check

* Add Startup Version Check and Service Naming

* make errormessage private

* Add Comments about VersionedClient

* Linting

* Remove Checks that refuse outgoing connections

* Remove release cmd

* Add Release Script

* Linting

* Update to use correct Values

* Move vars private and set minimum default versions for testing builds

* Remove VersionedClient

* Better Error Handling and naked return removal

* Straighten the Regex and string conversion

* Change Check to allows testplanet and storj-sim to run without the
need to pass an LDFlag

* Cosmetic Change to Dashboard

* Cleanup Returns and remove commented code

* Remove Version Check if no build options are passed in

* Pass in Config Values instead of Pointers

* Handle missed Error

* Update Endpoint URL

* Change Type of Release Flag

* Add additional Logging

* Remove Versions Logging of other Services

* minor fixes

Change-Id: I5cc04a410ea6b2008d14dffd63eb5f36dd348a8b
2019-04-03 21:13:39 +02:00
JT Olio
f96724df91 Forward port Bill and Stefan's changes to master (#1590)
* timeout calculation

* Initial Migration Draft to wipe the network gracefully (#1578)

* Initial Migration Draft to wipe the network gracefully

* Change from DATE to UNIX Timestamp (INT)

* SQL Derps

* Remove Satellite Migration - gets done manually

* Move from possible STRING to INT Vars

* Add TestTable for Migration Tests

* Adjust Test data

* Update TestData Part 2

* Update Wipeout to 28th 00:00

* merge spelling
2019-04-03 19:17:29 +02:00
Matt Robinson
48e7271ff2
Add ca-certificates to satellite dοcker-image so mail works (#1653) 2019-04-03 12:44:40 -04:00
Nikolay Yurchenko
48c4ac1260
Storj branding changes (#1655) 2019-04-03 19:25:18 +03:00
Bryan White
fe476fdcf1
extension serialization (#1554) 2019-04-03 17:03:53 +02:00
Natalie Villasana
e2a43c3fb6
adds defer close for node conn in audit verifier (#1634) 2019-04-03 09:42:24 -04:00
Maximillian von Briesen
6028d8c3de
Fix repair order limit creation (#1650)
* fix repair order limit creation (use piece size instead of share size)
* fix counter in CreateGetRepairOrderLimits
2019-04-03 09:17:32 -04:00
JT Olio
9af4f26d43 libuplink changes for public usage (#1568)
Co-authored-by: paul cannon <thepaul@users.noreply.github.com>
Co-authored-by: Kaloyan Raev <kaloyan-raev@users.noreply.github.com>
2019-04-03 11:46:21 +03:00
Cameron
34a439a99c
Delete accounting raws after rollup (#1646)
* enable deletion of raws

* assert deletion of raws during TestRollupRaws

* add some unit tests for DeleteRawBefore
2019-04-02 23:55:24 -04:00
paul cannon
e4a70e3fac
plumb EncryptionScheme, RedundancyScheme through to buckets (#1638)
We want to use those fields in the bucket-level Pointer objects as
bucket defaults, but we need to be able to get at them first.

I don't see any strong reason not to make these available, except
that it was kind of a pain.
2019-04-02 15:15:31 -06:00
Nikolay Yurchenko
5134766d57 Web usage report markup (#1409) 2019-04-02 22:33:03 +03:00
Egon Elbre
e016105a72 update some packages so we don't depend on launchpad (#1644) 2019-04-02 21:43:24 +03:00
Jess G
d51bdf14df
project usage limiting (#1561)
* reorg uplink cmd files for consistency

* init implementation of usage limiting

* Revert "reorg uplink cmd files for consistency"

This reverts commit 91ced7639bf36fc8af1db237b01e233ca92f1890.

* add changes per CR comments

* fix custom query to use rebind

* updates per convo about what to limit on

* changes per comments

* fix syntax and comments

* add integration test, add db methods for test

* update migration, add rebind to query

* update testdata for psql

* remove unneeded drop index statement

* fix migrations, fix calculate usage limit

* fix comment

* add audit test back

* change methods to use bucketName/projectID, fix tests

* add changes per CR comments

* add test for uplink upload and err ssg

* changes per CR comments

* check get/put limit separately
2019-04-02 11:21:18 -07:00
Natalie Villasana
2aeab10d50
skips flaky audit verifier test (#1645) 2019-04-02 13:44:33 -04:00
Simon Guindon
d3885b7b78
Change crypto/rand to use math/rand in pgutil (#1589)
* Added retry logic and tests for handling if crypto/rand fails.

* Added retry logic and tests for handling if crypto/rand fails.

* Fixing linting error.

* Removing use of `crypto/rand` for use of `math/rand`.

* Changing code comment about why we ignore this error.
2019-04-02 12:52:25 -04:00
Alexander Leitner
59b3eb190e
API endpoint checking object and segment health from the satellite (#1546)
* Generate protobuf for checking object and segment health

* Initial cmd client for health check

* Register the health inspector on the satellite peer
2019-04-02 10:55:58 -04:00
Simon Guindon
7f5e4ba658
Added Travis CI build status icon. (#1636) 2019-04-02 10:34:31 -04:00
Nikolay Yurchenko
ab1371d6ad Web forgot password flow markup (#1627) 2019-04-02 16:38:57 +03:00
Natalie Villasana
c7c6d3d597
skips flaky TestGetShareTimeout (#1635) 2019-04-02 09:16:13 -04:00
Egon Elbre
fba9a5f945 migration tests for storagenodedb infodb (#1628) 2019-04-02 09:54:09 +02:00
Natalie Villasana
42a141965d
fixes infinite loop in audit TestVerifierHappyPath (#1617) 2019-04-01 17:27:07 -04:00
Michal Niewrzal
f80750693c Store bandwidth from orders on satellite (#1586) 2019-04-01 16:14:58 -04:00
Kaloyan Raev
034c283eda Set last_contact_success when a node is initially added to overlay cache (#1632) 2019-04-01 12:42:06 -06:00
Cameron
6d43832c4f
record bucket data into bucket_storage_tally table (#1595)
* add MetadataSize to stats

* add logic for accumulating bucket stats in calculateAtRestData

* rename stats to BucketTally, move to accounting package

* define method on accountingDB for inserting bucketTallies

* insert bucketTallies into bucket_storage_tally table
2019-04-01 09:42:17 -04:00
Nikolay Yurchenko
42aad35b07 Moved templates and styles for Login, Register and Navigation (#1624) 2019-04-01 14:24:45 +03:00
Kaloyan Raev
6aa50529ff Add Fadila to CLA list (#1623) 2019-04-01 12:33:37 +02:00
littleskunk
43ef0eb4c3
Don't crash on audit and repair failures (#1622)
* Fix satellite crash on repair

(cherry picked from commit cabf6c9f97780f900d76e2388ffa54b916f14528)

* Fix satellite crash on audit

(cherry picked from commit 9da67488c4b36a378f346fbb27651316284b0f36)
2019-04-01 11:16:17 +02:00
Stefan Benten
c50a21d4cf
Fix Dashboard calculation for used space and bandwidth usage (#1615)
* Fix Dashboard calculation for used space and bandwidth usage

* Copy+Paste Err
2019-03-30 14:16:08 +01:00
Egon Elbre
872bd5d7c1
Remove pointerdb.Server (#1609) 2019-03-30 13:21:49 +02:00
paul cannon
3b12b5e85c
non-racy CreateEntryIfNotExists() (#1611)
This changes semantics slightly! with this change,
CreateEntryIfNotExists() will do a cache Update with every node passed
in, whether it exists or not. Update() already does a race-free upsert
operation, so that change removes the problematic race in
CreateEntryIfNotExists(). As far as I can tell, this semantic change
doesn't break any expectations of callers, and shouldn't affect
performance in a significant way, as we already have an awful lot of
round-trips to the db either way. But if I've misunderstood the
intention of the method, someone ought to catch it during review.
2019-03-29 15:07:07 -06:00
paul cannon
857edee485 assert.NoError() -> require.NoError() in kademlia tests (#1608) 2019-03-29 20:16:59 +02:00
Yehor Butko
66a7d6bf3a
Satellite password check simplification (#1610) 2019-03-29 19:42:20 +02:00
Egon Elbre
63737e350f
Delete psserver and unused mocks (#1605) 2019-03-29 16:40:06 +02:00
aligeti
ea22411cac
overlaycache's DB access modified to use transaction (#1596) 2019-03-29 09:46:17 -04:00
Egon Elbre
9dbc7c719e
fix error handling in GetByProjectID (#1604) 2019-03-29 14:55:05 +02:00
Egon Elbre
de15a4fdcf
remove utils.CombineErrors and utils.ErrorGroup (#1603) 2019-03-29 14:30:23 +02:00
Yaroslav Vorobiov
0ce6d4ab81
Project api auth update (#1572) 2019-03-29 14:13:37 +02:00
Kaloyan Raev
ddce5a4bf8
Fix incorrect usage of NodeID and ErrNodeID in PieceID impl (#1601) 2019-03-29 13:13:03 +02:00
Yehor Butko
e68c3e9c0d
Satellite console team members fetch bug (#1540)
* fetch project memebrs after project creation

* fetch fixed
2019-03-29 12:46:42 +02:00
Kaloyan Raev
9fb99c8484
Ensure serial number is not saved if error during order limit creation (#1602) 2019-03-29 11:53:53 +02:00
Kaloyan Raev
f9ba935286
Merge overlay_cache_nodes into nodes table (#1581) 2019-03-29 10:53:43 +02:00
Egon Elbre
be06fdfd6c Create orders.Service (#1593) 2019-03-28 22:09:23 +02:00
Bill Thorp
40e7fae564 not overflow int32 (#1594) 2019-03-28 13:08:32 +02:00
Jennifer Li Johnson
b3a8beb1bc
Updates accounting tables models and migration (#1584)
* go gen

* undo changes to bucket usage

* update locked

* spacing

* moves changes to migration v11

* minor changes to fix lint and test err

* change sql to fix errs
2019-03-27 22:46:49 -04:00
JT Olio
0a0984707c
clarify who fills in the code review checklist (#1555)
Change-Id: Ifabc09c6ed8dd10fcda6b0298d127b19d76e4674
2019-03-27 17:19:22 -06:00
Stefan Benten
7336e87e1c
Fixing Check for a writable folder of the identity tool (#1587)
* Fixing Check for a writable folder

* Removing fmt statements

* Check the folder not the file
2019-03-27 23:20:10 +01:00
Cameron
cac55a29e4
Add used egress/ingress to storage node dashboard (#1565)
* add egress and ingress to StatSummaryResponse

* print egress and ingress to storagenode dashboard
2019-03-27 15:44:18 -04:00
Yehor Butko
86bf3dee9f
V3-1307 combine first and last name to full name (#1569)
* V3-1307 combine first and last name to full name
2019-03-27 14:33:32 +02:00