this was just supposed to add parallel uploads/downloads
and it does do that, but i then found a bunch of bugs
with respect to path handling that i thought i had under
control. oops.
so this adds a ton of tests and tries to make the logic
in ulloc to be more consistent. almost all of the actual
file handling bits and knowledge happens in cmd_cp now
where it should belong.
additionally, the s3 command has the behavior that if your
bucket has the file s3://bucket/file, then executing
s3 ls s3://bucket/fi returns nothing. this change makes
uplinkng match that behavior even if i don't personally
like it.
a big portion of the weirdness is the concept introduced
that i've named "directoryish", which intends to capture
the behavior that if a user copies a file to that location
then the base name of the source should be appended on
rather than a direct copy. this concept is entirely a
based on the string value and not the actual filesystem
state. hence, the cp command is responsible for checking
if local paths are actually a directory, and adding a
trailing slash if necessary to make them "directoryish".
additionally, the empty key for a bucket and the empty
string for local paths are considered "directoryish".
Change-Id: I9120d18616fd813b29ff81beed4f5993caa99fb6
After migrating to eslint some errors were disabled to make it easier to
migrate.
This enables all the lint rules and treats all warnings as a build
failure. Similarly, CI won't automatically try to fix mistakes.
Change-Id: I80f808af026fc51bed90421b3b24737994a52094
When we can't complete an audit or repair, we need more information about
what happened during each individual share/piece download.
In audit, add the number of offline, unknown, contained, failed nodes to
the error log. In repair, combine the errors from each download and add
them to the error log.
Change-Id: Ic5d2a0f3f291f26cb82662bfb37355dd2b5c89ba
The Usage Report page referenced a nonexistent stylesheet and improperly
imported a font that does not exist in this repository. This change
resolves those issues.
Change-Id: I78006d4a652fd1e32dd4dd146b2287f58a7659d6
To optimize memory consumption we where consuming
segment data during processing results from delete
query. Turns out that there is a chance that query will be
rolled-back if something will go wrong while reading
results. In such case its possible to delete pices but
object/segment will be still in DB.
This change removed piece deletion from problematic
place. Pieces are still deleted in batches but are not
limited at the moment. To avoid memory issues object
deletion batch was decreased.
Change-Id: Icb3667220f9c25f64b73cf71d0cf3fdc7e5107c5
Rename the entities in the endpoint paths from singular after we have
agreed that it matches with API design best practices.
Change-Id: I5bdc086c64c6f93e45a13436b65863fc3e8d864e
This change adds a NOT NULL constraint to the created_at column in the segment table.
All occurrences of CreatedAt as a pointer are changed to non pointer version (metabase, segment loop, etc)
Change-Id: I3efd476ebd1edd3327b69c9223d9edc800e1cc52
MFA is complete and we are good to enable it in production. This change
removes the flag that disables MFA by default.
Change-Id: I2f985ae501171bdab505d664b43c8cfc248bad8d
After migrating to eslint some errors were disabled to make it easier to
migrate.
This enables all the lint rules and treats all warnings as a build
failure. Similarly, CI won't automatically try to fix mistakes.
Change-Id: Iff9c4a59401900fc395cd566dd328f3a9c688a12
We need a way to delete whole part. This especially
needed for uplink multipart API to do cleanup after
aborted or failed part upload.
Test will be added when uplink part will be merged.
Change-Id: I9ba69a49e1adcdce0f42dd3a76f938fcf931155a
After migrating to eslint some errors were disabled to make it easier to
migrate.
This enables all the lint rules and treats all warnings as a build
failure. Similarly, CI won't automatically try to fix mistakes.
Change-Id: I6d5ef39d476a38dd9a6ab98e3fb94da4c3ab2e57
tslint has been deprecated so it's nice to switch to eslint.
Currently this uses a minimal eslint, to get things up and running.
node-sass requires C which does not work nicely on all platforms.
Change-Id: I3ca9adf2971475c009e541652b7637c18ad960f4
This change adds dedicated methods on metabase.Pieces to be able to add, remove pieces and also to check duplicates.
Change-Id: I21aaeff40c017c2ebe1cc85a864ae546754769cc
tslint has been deprecated so it's nice to switch to eslint.
Currently this uses a minimal eslint, to get things up and running.
node-sass requires C which does not work nicely on all platforms.
Change-Id: Icf12e375293e052befc2d0daeb23d73697ed6ded
Added includeMetadata parameter which represents if metadata should be included in response
by default true, in case of new uplink version - ObjectIncludes will be used instead.
Change-Id: I2f8d3b4cc354cd655f8093bbbebe0e3c2ae14e6f
Error from joining loop should not restart satellite. This will be the
same error like for loop itself. In the same way we are handling joining
error for other services that are using segment loop.
Change-Id: Idf1035ef7f78462927bd23989ed8a4ee5826c49e
tslint has been deprecated so it's nice to switch to eslint.
Currently this uses a minimal eslint, to get things up and running.
Change-Id: If0ca6413b534b0aa15f6130a0a809e002d817356
Added ability to regenerate MFA recovery codes to account settings.
Appropriate button and popup were added.
Change-Id: Ie7d42e7e13e868f3b0b8b7d5db7eacc9e8206aaf
In case when number of deleted segments in a single
batch was multiplication of opts.DeletePiecesBatchSize it
was possible to finish deletion of objects to early.
Change-Id: I9181d4a3c64053d9df46a11a6e0cd22153c73ee9
Added Enter MFA passcode step to login after first attempt.
Connected client-side and server-side.
Change-Id: I4482afde50172bbf2c598aa01bbd220763b4f6d1
We made decision to avoid satellite shutdown when segment loop
will return error. Loop still can reeturn error but it will be logged
and we will make monitoring/alert around that error.
Change-Id: I6aa8e284406edf644a09d6b1fe00c3155c5430c9
Error message for attempting to activate an already-activated
account is removed from its own page and incorporated into the
login page in an identical manner to the message that appears
upon successful activation.
Change-Id: I29cd2685a7808fa71d34a439c86a38eb5fc3e199
It turns out that some DB methods are not
passing context to to lower levels even if have
context as a parameter.
Change-Id: I82f4fc1a47c362687a91364d85e4468057e53134
gracefulexit
With the effort to move audit related data into reputation store, this
PR updates gracefulexit endpoint to use reputation service to get a
node's audit score
Change-Id: Iad93ea689ad67ff9c57c7be16687e21e715fab7a
