Currently the paths were set relative to the root of the module,
however the code did not ensure that we are running relative to the
module directory.
Also, ensure typescript output corresponds to our styling.
Change-Id: I2b3cbd4ea8f2615e35c7b58c6fb8851669c47885
The collector calls the Delete() method on the pieces
which returns an error which is wrapped by many error classes.
Delete() method is using Stat() from
1aec831d98/storage/filestore/dir.go (L328)
under the hood.
os.IsNotExist(errors.Unwrap(err) will always be false unless
errors.Unwrap(err) is called multiple times till it gets to
the core os.ErrNotExist. Here is a test case to explain better:
func TestABC(t *testing.T) {
classA := errs.Class("A")
classB := errs.Class("B")
wrappedError := classB.Wrap(classA.Wrap(os.ErrNotExist))
require.True(t, os.IsNotExist(errs.Unwrap(wrappedError)))
require.True(t, os.IsNotExist(errors.Unwrap(wrappedError)))
}
Using errs.Is() seems to resolve this even without unwrapping the error:
func TestABC(t *testing.T) {
classA := errs.Class("A")
classB := errs.Class("B")
wrappedError := classB.Wrap(classA.Wrap(os.ErrNotExist))
require.True(t, errs.Is(wrappedError, os.ErrNotExist))
require.False(t, errs.Is(wrappedError, os.ErrExist))
require.False(t, os.IsNotExist(wrappedError))
}
Does not resolve the collector issue here but enhances it:
https://github.com/storj/storj/issues/4192
Change-Id: Ifb75dd15b54c1e1a5e23f6eba2d621d64874a5cc
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket.
This change adds integration with testplanet which makes writing
unit tests possible.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: I7b335c5dafa8cffe265c56b75d8c8f8567580893
Added server-side encryption warning button to buckets page.
It can be dismissed and will show up again only on next user login.
Issue:
https://github.com/storj/storj/issues/5118
Change-Id: Iec5de91974ad5177a322b205f6b361c86f06923e
Reverts two of the changes in commit 119e61fcb because the timing of the
new docs deployment is changing.
Change-Id: Iaf804a41073d50700f67b153ef088631edefe6e5
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket. This this initial change to add such
service. Added service is joining segment loop and collects all
bloom filters.
Sending bloom filters to the bucket will be added as a subsequent
change.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: I2551723605afa41bec84826b0c647cd1f61f3b14
This change adds the following endpoints:
- projects/apikeys/{id}: returns a paged list of API keys for the
project specified by the given ID
- apikeys/delete/{id}: deletes the API key specified by the given ID
Additionally, the API Go code generator has been given the ability to
process unsigned integer parameters.
Change-Id: I5ff24e012da24a3f06bea1ebb62bae6ff62f951a
2.7 natively supports the Composition API, which makes transitioning to
Vue 3 easier to do step-by-step.
Converted ::v-deep selectors to :deep() to fix new build warnings.
Change-Id: I21b34e10f55eaf9dcdad87729f02f3b0d723b6b1
Add the users current wallet balance to the endpoints for claiming and listing storjscan wallets. Also prevent a user with a claimed wallet address from claiming a new wallet.
Change-Id: I0dbf1303699f924d05c8c52359038dc5ef6c42a1
The anchor tag for these links is changing, so we need to update the UI
for them to work. Also remove trailing slashes from docs.storj.io links.
Change-Id: Ic4682dae76c44cf803b16ebfe7352405520c4307
This is to update all projects to have a public_id if they do not have
one.
github issue: https://github.com/storj/storj/issues/4861
Change-Id: Icfa42b62e15ca75d3c04a0aab48a3c3b0f3a9d6e
method similar to metabase.DeleteObjectExactVersion which will delete last committed object
Closes https://github.com/storj/storj/issues/4872
Change-Id: Ia9f8c227dc59575bf8ed297886b35536097028b4
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket. This change is extending satellite binary
with appropriete command.
New GC service for collecting bloom filter will be a subsequent
change.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: Ibc03e119c340919cf468fc1f5a4f3d187bb3a5a1
As a reminder: latest clingy removed the requirement of having custom context (which made the usage of context.WithValue harder) and uses simple context instead.
Clingy saves the stdin/stdout/stderr to the context (earlier to separated context type) to make it available for unit testing.
Change-Id: I8896574f4670721de43a577cd4b35952e3b5d00e
Adds USDollarsMicro currency to the billing DB which support fraction of a cent with decimal places for better billing amounts accuracy.
Change-Id: Id07dfae104d94e27c7b22ab8f5781010e16c4c8e
Removed batch insert of payments since they do not guarantee order. Order of payments sent to the payments DB is important, because the billing chore will request new payments based on the last received payment. If the last payment inserted is not the last payment received, duplicate payments will be inserted into the billing table.
Change-Id: Ic3335c89fa8031f7bc16f417ca23ed83301ef8f6
Improve styling for lists in `customHtmlDescription` for alternate
signup landing pages in `registrationViewConfig`.
Change-Id: If20831044099a1183e5dc8b41668bf2f68c272a2
We would like to have separate process/command to collect bloom
filters from source different than production DBs. Such process will
use segment loop to build bloom filters for all storage nodes and
will send it to Storj bucket. This change is just initial code with peer
which will be used to build this new process.
Updates https://github.com/storj/team-metainfo/issues/120
Change-Id: I10a52b74865ce8ec4c29b7c6a2836f9232620422
Add new protobuff message to store retain filters before sending them
out to storagenodes.
This allows the retain filters to be generated by a separate command on
a backup of the database.
Change-Id: I8a2892d9d34bed1dbd6e5a7ded000dcdea7ec061
We are preparing to use object versions internally and to do
that we need to prepare different parts of the system to handle
object versions different than '1'. This change adjust code
responsible for server-side move and copy.
What was done:
* begin methods for move and copy are now using GetObjectLastCommitted
to find object
* results from begin move and copy operation contains now version to
be able to map object correctly with finish operation
* begin methods are putting version into satellite stream id and
finish methods are using this version as parameter instead hardcoded
value
Fixes https://github.com/storj/storj/issues/4867
Change-Id: I1380911279c21e10a3fff0342793efd2e73eafad
Implemented share bucket feature.
Refactored share object modal a bit (has to be refactored entirely).
Issue:
https://github.com/storj/storj/issues/4945
Change-Id: Icefd4bfe3eef9173ae824eea44d30450acde8044
This concludes the saga that began with commit c053bdbd, migrating away
from using gob-encoded big.Float values in the database to using
integers with an implied number of decimal places.
Nothing is using or accessing or expecting the _gob columns to exist
anymore. The transition code and the migration chore are gone. All that
remains is to drop the columns.
Change-Id: I9b15ee52f7781510a6dc91cf7c54f7f9022b1210
update responsiveness in billing pages to match figma designs including added billing header scrolling for smaller windows
Fixes https://github.com/storj/storj/issues/5033
Change-Id: I94fc836f95e3985d6b675d3495961ec703e43044
Currently we are tracking loop time with RunOnce method
metrics but it may give inaccurate values if loop if waiting
for observer for a ver long time. We had such case with GC
which is only loop observer in GC peer and its joning loop
every 5 days.
Change-Id: I08546c912e00c3641488de6a5d75948fe75c8e99
Today each storagenode should have a port which is opened for the internet, and handles DRPC protocol calls.
When we do a HTTP call on the DRPC endpoint, it hangs until a timeout.
This patch changes the behavior: the main DRPC port of the storagenodes can accept HTTP requests and can be used to monitor the status of the node:
* if returns with HTTP 200 only if the storagnode is healthy (not suspended / disqualified + online score > 0.9)
* it CAN include information about the current status (per satellite). It's opt-in, you should configure it so.
In this way it becomes extremely easy to monitor storagenodes with external uptime services.
Note: this patch exposes some information which was not easily available before (especially the node status, and used satellites). I think it should be acceptable:
* Until having more community satellites, all storagenodes are connected to the main Storj satellites.
* With community satellites, it's good thing to have more transparency (easy way to check who is connected to which satellites)
The implementation is based on this line:
```
http.Serve(NewPrefixedListener([]byte("GET / HT"), publicMux.Route("GET / HT")), p.public.http)
```
This line answers to the TCP requests with `GET / HT...` (GET HTTP request to the route), but puts back the removed prefix.
Change-Id: I3700c7e24524850825ecdf75a4bcc3b4afcb3a74
Currently, the satellite tracks connectivity information about all nodes
that have contacted it, even if we have never successfully contacted
the node back.
This behavior was leveraged during a security audit to create hundreds
of thousands of "junk nodes" in the nodes table on one satellite, which
affected performance of queries such as node selection.
With this change, we should no longer track information about nodes that
have never been successfully contacted.
Note that it will still be possible to cause the creation of "junk
node" entries in the db; the attacker just has to set up individual
publicly-routable IP+port pairs for each node as it is created, so it
can respond to a PingBack.
Change-Id: Ibb6da6cc908fd4fc85aae1ba00313ba2738409ab
This change fixes the duplicate navigation error on the new billing screen.
see: https://github.com/storj/storj/issues/5096
Change-Id: Ie0512de806835acfc29e86182e4c49ced6355383
Decode the JSON input string to its corresponding unicode
decoding if the special byte order mark is present.
Fixes https://github.com/storj/storj/issues/4950
Change-Id: If91bac22590c89b35c58bf54f6d3bdb8a67d7a4f
Metadata validation for CommitObject request was placed in a wrong
place. There is a case (old uplink) where encrypted key is bundled
inside encrypted metadata bytes and we need to extract it before we
can validate it. This change moves metadata validation to a place where
we are sure we have encrypted metadata and encrypted metadata encrypted
key ready to be checked.
"Run Versions Test" is covering this case and it was failing without
this change.
Change-Id: Ib709ad901fbb3fa4865a393195b7b3f4c0d87e7a
For object Version in different places we are using different types.
Satellite StreamID is using int32 but metabase accepts int64. Metabase
type is correct one and we should align other places with it.
As a small addition this change is also passing version correctly
between requests instead of using hardcoded value.
Change-Id: I63634d73c0a48c009e4db5f203ff18b7f3218b02
Updated metabase.UpdateObjectMetadata method to update set metdata always for last committed object
Closes https://github.com/storj/storj/issues/4870
Change-Id: I060683e31efcaf3e2531fea143cf0567e5ff5f73
We couldn't use environment variables safely to configure storagenode, since we introduced the embedded updater.
For example STORJ_DEBUG_ADDR=localhost:11111 would try to set debug port 11111 for both the storagenode and storagenode-updated, causing port conflict.
This small change enables to configure storagenode-updater with STORJUPDATER_... environment variables.
Tested with creating custom image and installing to my own storage node.
Change-Id: I6b0a601a4dc63d2d1ff3c191ae89981434e55c30
Sessions now expire after a much shorter amount of time, requiring
clients to issue API requests for session extension. This is handled
behind the scenes as the user interacts with the page, but once session
expiration is imminent, a modal appears which informs the user of his
inactivity and presents him with the choice of loging out or preserving
his session.
Change-Id: I68008d45859c814a835d65d882ad5ad2199d618e
after redirecting from bucket page to upload page if we have no files/folders
related to this passphrase but have other objects in bucket we show closable
reminder that we support multiple passphrases per bucket
Change-Id: I6420aedd5605100e4aa35b598771e5298e251f91
When enconding structs into JSON, byte slices are marshalled as base64
encoded string using the base64.StdEncoding.Encode():
ea9c3fd42d/src/encoding/json/encode.go (L833-L861)
We, however, expect API Secrets to be encoded as base64URL, so when
an marshalled secret (with byte slice type) is added to the multinode
dashboard, it fails with `illegal base64 data at input byte XX`.
This change changes the type of APISecret field in the
multinode/nodes.Nodes struct to use multinodeauth.Secret type instead
of []byte.
multinodeauth.Secret is extended with custom MarshalJSON and
UnmarshalJSON methods which implement the json.Marshaler and
json.Unmarshaler interfaces, respectively.
Resolves https://github.com/storj/storj/issues/4949
Change-Id: Ib14b5f49ceaac109620c25d7ff83be865c698343
This change tracks signup captcha scores in the signup_captcha column in the users table.
It slightly modifies the captcha verify method to return both the score and success.
see: https://github.com/storj/storj/issues/5067
Change-Id: I7b3993e44958cfcf179806c7df19d6887fe3eda9
