Commit Graph

199 Commits

Author SHA1 Message Date
Stefan Benten
a71c908c1e
cmd: add ca-certificates to Docker images (#3986) 2020-12-08 01:38:33 +01:00
Bill Thorp
47e008b714 cmd/uplink: Write to AWS Credentials file more safely from uplink access register
Jeff provided feedback on https://review.dev.storj.io/c/storj/storj/+/3176 after
the changeset was already merged.  I attempt to address that feedback here.

Change-Id: Ibc7dba3e4e2c73736042fe4b4ee49ce679ba7f44
2020-12-01 17:47:14 +00:00
Bill Thorp
65919f9f7d cmd/uplink: add --aws-profile flag to uplink access register
Added flag to append a new profile to ~/.aws/credentials using
the provided profile name.  This is handy for the AWS CLI, so
you can do things like 'aws configure get aws_access_key_id --profile=me'

Change-Id: I0469a18ca76e078624ed455a06bd7aabd95a1b97
2020-11-30 09:39:01 -05:00
Jeff Wendling
54ae9b040e cmd/uplink: allow public access registration
Change-Id: I1316f10158d8a466bc80d6548ad0773016bfb916
2020-11-18 17:40:30 -05:00
Bill Thorp
5fe3d2dea7 cmd/uplink Allow use of named accesses in uplink register
Previously uplink register only accepted a fully serialized access grant.
This is kind of annoying, I changed it so that it could also use access names.

Change-Id: If6d4d1baa8d4fb3d87fdedb895d459fa12743f1a
2020-11-18 12:23:57 -05:00
Isaac Hess
e32466696a cmd/uplink: add access register command
This command allows you to register your access with a separate service
(hosted gateway).

Change-Id: I67200cd4553fd41e9b1a56cca7ccc6553136a76a
2020-11-10 15:25:57 -07:00
Egon Elbre
2268cc1df3 all: fix linter complaints
Change-Id: Ia01404dbb6bdd19a146fa10ff7302e08f87a8c95
2020-10-13 15:59:01 +03:00
Monty Anderson
7710e083d4
cmd/uplink: update CLI format (#3773) 2020-09-23 12:54:00 +02:00
Yingrong Zhao
af773ec8a6 cmd/uplink: use DeleteBucketWithObjects for bucket force deletion
This PR updates `uplink rb --force` command to use the new libuplink API
`DeleteBucketWithObjects`.
It also updates `DeleteBucket` endpoint to return a specific error
message when a given bucket has concurrent writes while being deleted.

Change-Id: Ic9593d55b0c27b26cd8966dd1bc8cd1e02a6666e
2020-09-02 16:39:20 +00:00
Egon Elbre
94a09ce20b all: add missing dots
Change-Id: I93b86c9fb3398c5d3c9121b8859dad1c615fa23a
2020-08-11 17:50:01 +03:00
Egon Elbre
080ba47a06 all: fix dots
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
2020-07-16 14:58:28 +00:00
Egon Elbre
29ccda5d47 cmd/uplink/cmd: remove spf13/cast dependency
Change-Id: I8013014b0b37966128e673155d851705ee1d43db
2020-07-13 11:10:48 +00:00
Egon Elbre
c40d5043e1 cmd/uplink: pass useragent flag to library
Change-Id: I8cf12c978c01eca968086560be0c28809eb2ba7d
2020-07-03 16:41:35 +03:00
Isaac Hess
cad21f11e5 cmd/uplink: Add revoke command
Change-Id: I8ae74da5c9171429b970d76598793a3227ed4d11
2020-06-22 13:50:07 -06:00
JT Olio
2bd9067ad5 cmd/uplink: tighter linksharing security
when a user runs `uplink share`, they get a bunch of results back,
given their configuration and existing access. one of the results
is a URL for in-browser sharing and hosting of the file.

first off, we want to make sure this URL is read only. we want to
avoid a situation where someone post this URL to some public
location, not realizing the access allows writes or deletes. if
a user really wants a URL with write/delete access, they can
construct it themselves.

secondly, we want to make sure the url is sharing a single path or
path prefix. having a url for multiple paths/path prefixes
indepedently again can be constructed of course, but should not
be the default behavior

Change-Id: I2ca2ebeea9f1c7d4bfbd7a437a32dc7a3b2a32cc
2020-06-16 15:02:53 -06:00
Egon Elbre
34db4a80fd ci: fix staticcheck failures
Change-Id: I176fb24214755a1940a0a1a4e9cc8e39f184870b
2020-06-05 13:15:34 +00:00
Michal Niewrzal
056ae7ffa8 scripts/test/rollingupgrade: set correct satellite address for imported
access

Addition: use always latest gateway release

Change-Id: I5e5231e6da4b6f7900cb71bb6e227901474270ea
2020-06-04 15:04:10 +00:00
Michal Niewrzal
79a562c3ed cmd/uplink/cmd: improve uplink remove bucket command with force flag
* don't stop on single delete failure
* display number of successful and unsuccessful deletes

Change-Id: Id7f29055f250aad515e46622c2bfd3596055b93a
2020-06-01 07:56:14 +00:00
Michal Niewrzal
84892631c8 private/testplanet: remove old libuplink from testplanet
Change-Id: Ib1553f84d0b3ae12a5b00382f0f53357b6a273e2
2020-05-28 13:50:23 +00:00
Michal Niewrzal
a4c19b3359 cmd/uplink/cmd: remove port detection when selecting satellite while
setup

We don't have default ports as a part of configuration anymore because
satellite-addr flag was removed.

Change-Id: Ibf9fc4b399beaf51ebb9461de2d8994a322f9686
2020-05-26 13:25:51 +02:00
Michal Niewrzal
340700eda9 cmd/uplink/cmd: remove old libuplink from Uplink CLI code
Change-Id: I38444a67e3bdec5f63c56895b5fc362464e7dce1
2020-05-22 09:08:58 +00:00
Bill Thorp
bcb867f1de cmd/uplink: add UserAgent flag for partner value attribution
This changeset allows a user agent string to be set during uplink setup, which
is thereafter used for partner value attribution.  EG

uplink setup --client.user-agent ”MyCompany”

Change-Id: Iefa8755fccc06acb8a303a342b943cece44a81f7
2020-05-22 07:39:14 +00:00
Jeff Wendling
f93d764998 cmd/uplink: properly path escape shares
Fixes #3829

Change-Id: I0f900a392239fcc064cea968fc7a5bc373ee26a1
2020-05-04 17:00:25 +00:00
Yingrong Zhao
0b80d60491 cmd/uplink: disable tlemetry if user opt out
Change-Id: I793e6428d04e4f422d1f8292e99b233c70d9dd6e
2020-05-04 14:46:49 +00:00
Yingrong Zhao
c5309a3f91 cmd/uplink: set sample rate for tracing to be 1 when tracing is enabled
when tracing is enabled, we should also set sampling rate to
a non-zero value. For now, we will set it to 1.
Uplink CLI users should be able to override it with the sample
flag.

Change-Id: I8bcf514fb14c2a1c4349b7957dd24ec23e4a85e5
2020-04-28 20:15:28 +00:00
Yingrong Zhao
9b4a3f8fcc cmd/uplink: use tracing.enabled flag
Previously we are using tracing.sampled to be the switch for turning on/off tracing.
However we would like to separate sampling rate from being the switch,
so we can set sampling rate to be 0 but still intialize tracing for
satellite and storagenodes

Change-Id: I27e6ba25ea6f6b612b4e1a57cf1301889ded41ec
2020-04-27 17:54:57 +00:00
Yingrong Zhao
0b8699bcb5 cmd: add prompt for enabling tracing during uplink cli setup
We want to make tracing to be opt-in.
For now, we will use `tracing.sample` as the toggle config to enable or
disable tracing and default to sample every traces from uplink cli.
If user wants to change the default sampling rate, they can do so by
using the `--tracing.sample` flag to override the default value

Change-Id: I6f25dac0f43024c50a8aaf6c549e6a514211f834
2020-04-21 20:57:10 +00:00
Michal Niewrzal
90bac51544 cmd/uplink/cmd: improve error message when config.yaml or --access flag
is missing

https://storjlabs.atlassian.net/browse/USR-78

Change-Id: I8221822d164279b85f24573df0ac05fcd6b3881e
2020-04-15 13:22:02 +00:00
JT Olio
e2d5b403e6 cmd/uplink: support --force (like awscli) for rb
Change-Id: If835c6dd08ee95e7c66ba7e4c7451cb3f0f95442
2020-04-14 18:10:54 +00:00
Egon Elbre
644df8dcdc private/version: minimal fix for tag-release.sh
Previous split to a storj.io/private repository broke tag-release.sh
script. This is the minimal temporary fix to make things work.

This links the build information to specified variables and sets them
inline. This approach, of course, is very fragile.

Change-Id: I73db2305e6c304146e5a14b13f1d917881a7455c
2020-04-01 13:46:45 +00:00
Michal Niewrzal
f0aeda3091 storj: remove from storj/pkg packages moved to storj/private repo
* debug
* traces
* cfgstruct
* process

Package `storj/private/version` will be removed as a separate change.

Change-Id: Iadc40faa782e6225513b28218952f02d9c240a9f
2020-03-24 09:56:29 +01:00
Egon Elbre
6a7571f73e cmd/s3-benchmark: move to storj.io/benchmark
Change-Id: Idca2b836bdf876ca28eb5cabc9bfae1d576e4a3e
2020-03-23 19:09:42 +02:00
Isaac Hess
8e274a5ce4 cmd/uplink: Enable telemetry on ctx
Change-Id: I26ffbbb72b9f2ca71b629ba12637c74c2aa072c3
2020-03-12 14:52:40 +00:00
JT Olio
e4d5addb0d cmd/uplink: add url-based link sharing
Change-Id: I83174b2d686be5e5a6910a343b507ab6157aa1f5
2020-03-10 10:15:42 +00:00
Michal Niewrzal
d7b5df70d3 cmd/uplink: remove unused flag
New API has limited number of options to configure at the moment. We
should remove unused flags from Uplink CLI and add if needed in the
future.

Change-Id: Icf3f3dadd43cb61a3b408b02d0762aef34425dbf
2020-03-09 13:44:46 +00:00
Michal Niewrzal
c20cf25f35 cmd: migrate uplink CLI to new API
Change-Id: I8f8fcc8dd9a68aac18fd79c4071696fb54853a60
2020-03-09 13:26:29 +00:00
Michal Niewrzal
cea4c25f53 mod: bump common and uplink version
Change-Id: Ia063d33c087dd91a46c008e154b078f11fa21527
2020-02-12 14:33:54 +00:00
Egon Elbre
d2fca76146 cmd/uplink/cmd: set exact argument counts
It was possible to call

   uplink cp a b c d e sj://bucket/something

Change-Id: I731da0da4530a3b3f8fbc569f363ba40cf84853a
2020-02-11 13:09:38 +00:00
Jeff Wendling
7999d24f81 all: use monkit v3
this commit updates our monkit dependency to the v3 version where
it outputs in an influx style. this makes discovery much easier
as many tools are built to look at it this way.

graphite and rothko will suffer some due to no longer being a tree
based on dots. hopefully time will exist to update rothko to
index based on the new metric format.

it adds an influx output for the statreceiver so that we can
write to influxdb v1 or v2 directly.

Change-Id: Iae9f9494a6d29cfbd1f932a5e71a891b490415ff
2020-02-05 23:53:17 +00:00
Egon Elbre
f237d70098 storagenode,satellite: use pkg/debug
Use debug.Server in storage node and satellite for customizing debug server.

Change-Id: I7979412376d028cadf29656d838ab94f18e2aa99
2020-01-29 16:30:31 -05:00
Fadila Khadar
3a0f6de66d cmd/uplink: list available accesses
Change-Id: I7748cc68cf21b514285579d15c1cab8329ac3d2b
2020-01-28 22:29:49 +00:00
Bryan White
f917fecc61 cmd/uplink: remove non-interactive mode from setup
Change-Id: I7934cfd0ef06e009a6ff38694ba0dd1806ffcdd8
2020-01-24 16:01:39 +00:00
Michal Niewrzal
416e5053a3 cmd/uplink: add note about share not-after/not-before flags formats
https://storjlabs.atlassian.net/browse/V3-3588

Change-Id: I43075155370c7e31ffee253b0f7169d7dee92c80
2020-01-24 12:42:36 +00:00
Bryan White
746c07cb91 cmd/uplink: improve usage for uplink subcommands
Positional args were missing from the usage strings.

Change-Id: I8458739577ff8d2b1c0503303023569da3d32806
2020-01-24 10:05:54 +00:00
paul cannon
cadd727df8 cmd/uplink: just a silly spelling fix
Change-Id: If3b5abdee9fa51fc9c23f2f2de9edcd7ab9ed039
2020-01-23 16:15:01 +00:00
Bryan White
fab58e9c12 cmd/uplink: hide advanced flags from output
Change-Id: I536af267c38e153aeea682fca4a74dc0ea2c42f0
2020-01-23 13:24:30 +00:00
Bryan White
d0041c94dc pkg/process: increase default log level to warn
Change-Id: I2ae089dcb584061627ff8254e7e34d7a6ff3a158
2020-01-22 14:31:28 +00:00
Ivan Fraixedes
80a6219c4b cmd/uplink: Create dir before saving config file
Setup command of uplink has to create the configuration directory just
before saving the configuration file for making it more robust than
creating in the initial state of the process.

When creating the directory at the beginning of the process leaves the
possibility to delete such directory during the setup process and leads
to a failure.

Ticket https://storjlabs.atlassian.net/browse/V3-3545

Change-Id: I30db0175e23a597e9675d267b4d7e25d5d4c5119
2020-01-22 13:36:09 +00:00
Bryan White
cb827cbe31 cmd/{uplink,gateway}: update cli copy "http" -> "https"
Change-Id: I47f46221ab3eb6065d382acfe75865abf09b1ce5
(cherry picked from commit 580a00561ae35b4830acc1f4dd990329df4d3c7b)
2020-01-21 11:35:15 +01:00
Bryan White
3b55b50eac cmd/uplink: Add ability to generate named accesses
Change-Id: I2cf58c1c41bfffa800949dc441a488c16a448375
2020-01-20 10:02:13 +01:00