Commit Graph

223 Commits

Author SHA1 Message Date
Egon Elbre
75ca01e381 cmd/uplink: fix importing access in Windows
Currently os.Create was leaving a file open causing atomic write file to
fail with access denied.

Also add a specific test for importing an access.

Change-Id: Id188bc480e795849ec7fdc72b1fc86433d76c47a
2021-04-19 15:11:39 +03:00
JT Olio
f3c8cac1ee storj/storj: more domain changes
Change-Id: I91ae3bc36be38c27269f9e6e678fa3616b8e4d53
2021-04-14 20:49:12 +00:00
JT Olio
afcc55fb47 storj/storj: some domain changes we can make now
Change-Id: I5622edea9dfcad788cac65e1754b331b9fed6154
2021-04-13 17:12:06 +00:00
JT Olio
2ddbaf1eb5 satellite/wasm: support restricting full access grants to paths
Change-Id: Id6d4fa41db068d32e7c0d542d9d8805fba927fc6
2021-03-26 19:49:21 +00:00
Michał Niewrzał
237782813b Merge remote-tracking branch 'origin/multipart-upload'
Change-Id: If6c5a450b238adab55d1e0dea67d01e5f5768a9f
2021-03-23 09:44:49 +01:00
Yingrong Zhao
bf4fdc7717 cmd/uplink: add --client.enable-quic flag
Change-Id: I19e81ecb98688fd0a36174fd06c363e9c40b3866
2021-03-19 01:24:16 +00:00
Fadila Khadar
cf0c88c679 cmd/uplink: fix no result when listing single file
uplink ls sj://bucket/file.ext gives a result if file.ext exists.

Change-Id: Ife741926acacf510e4e098009ce1724386c7522f
2021-02-26 11:43:02 +00:00
Fadila Khadar
8bbabe57db cmd/uplink: allow deletion of pending objects
Basic command for deleting a pending object by object key.

Change-Id: I47f68acf79cb0568a42a65b6d8195ca309b185a7
2021-02-25 14:23:56 +00:00
Fadila Khadar
e96ed289e5 cmd/uplink: add --pending for listing pending objects
Change-Id: I19c1965117c386f63e7b1c71ba70402d312329f0
2021-02-25 13:50:10 +00:00
JT Olio
e23f423107 uplink: add api key back to inspect
Change-Id: I9aa46f3910a76a81a399597d7512056b218e55af
2021-02-19 14:46:35 +00:00
JT Olio
1ac17cfac5 uplink: link.tardigradeshare.io URLs should have an /s/ right after the domain
should be
https://link.tardigradeshare.io/s/<access>/<bucket>/<path>

legacy URLs have the /s/ missing but a redirect is issued

Change-Id: Ic2a3dc092ff68d7706fd888a9fbfc27716877c6e
2021-02-09 14:38:22 -07:00
Caleb Case
22bc69ad6d
cmd/uplink: Add more details to access inspect
Change-Id: Ia7e600c214d6097b48b9c868532b8fbe21b0035c
2021-02-02 10:25:26 -05:00
JT Olio
6e2450ecf7 uplink cli: make uplink share --register output access keys again
this used to work prior to https://review.dev.storj.io/c/storj/storj/+/3466

Change-Id: I9f02410f0e3768f0df0c5309c3107609fcbf7742
2021-02-01 15:50:02 +00:00
Caleb Case
d2148edcdb
cmd/uplink/cmd/setup.go: Access Grant vs API Key error messaging
Provide a clearer error message to users who confuse the API Key with
the Access Grant and suggest the right command to them.

Change-Id: If73ae8cde140b68a19f4cfc3f59bb88a3b74c9c1
2021-01-26 14:09:59 -05:00
Isaac Hess
1e0d57bf01 uplink: Add timeout to RegisterAccess
We add a timeout for the http client used to register the access with
the auth service. We have a hard-coded common default for now.

Change-Id: I50207ad83c9221b7cb61f39310e24b140b95673b
2021-01-19 20:14:58 +00:00
Bill Thorp
c0bc2123f5 uplink/cmd: fix uplink access register ability to use saved accesses
Jens noticed that 'uplink access register' wasn't working with named
accesses.

This was because GetNamedAccess was harcoded to use inspectCfg, which
in the case of 'uplink access register' wasn't being bound to the
config file.

Change-Id: I49403b45af28ad33408cfc5ec6545a395f0f080d
2021-01-06 15:08:51 +00:00
Moby von Briesen
641f01ed42 cmd/uplink: Actually overwrite accesses in uplink import
Previously, we were trying to overwrite accesses, which is a nested map
in the uplink config, by calling viper.MergeWithConfig with a nested
map. While this works for keys that don't exist already, it does not
overwrite already existing keys. In order to do that, we need to call
MergeWithConfig with "accesses.<accessname> -> value" rather than using
"accesses -> <accessname> -> value".

Change-Id: I74d7a9decf2078cdf2ff440eaf24821e30474b53
2020-12-30 15:25:53 +00:00
Bill Thorp
f65a1e8c1c uplink: use output formats for 'uplink access register' instead of writing to AWS file
The current uplink access register method has the ability to write to AWS credential files.
This is caused issues with repeat usage in recent AWS CLI code, and there was concern that
it was an unstable solution.  This version instead specifies various output formats "env"
and "aws". "env" formats the text so that it can be used with 'export'.  "Aws" generates
"aws configure" commands to persist the credentials to the AWS credential files as the
previous version could.

Example usages:

Setting ephemeral evironment variables in bash:
export $(uplink access register $(storj-sim network env GATEWAY_0_ACCESS)
--auth-service http://localhost:8000 --format env)

Setting persistant configs via AWS CLI in bash:
source <(uplink access register $(storj-sim network env GATEWAY_0_ACCESS)
--auth-service http://localhost:8000 --format aws --aws-profile storjsim)

Change-Id: I5d78d6462a3537780af3717a298bb2bebf9c2799
2020-12-30 13:34:01 +00:00
Jennifer Johnson
7cccbdb766 cmd/uplink/share: add -- to flags referenced in help statements
Change-Id: I86cd08d51c5306effec14f338f37c53c2743d6b2
2020-12-22 17:03:00 +00:00
Bill Thorp
1b0424cad6 uplink/cmd: Export a RegisterAccess method.
Gateway-MT requires integration tests, which would be aided by having an
exported RegisterAccess() method in uplink/cmd.

To support this change, a little of the Uplink cmd logic was shifted around
and a method was made public.  I also normalized finding the access
between accessInspect and accessRegister.

Change-Id: I29369296521c2cc179e27233f5451b95f46109d8
2020-12-22 14:52:40 +00:00
Stefan Benten
7eab859030
cmd: ensure proper arch is used for docker container 2020-12-20 09:26:23 +02:00
Egon Elbre
12055e7864 all: minor cleanups
Change-Id: I4248dbe36a62a223b06135254b32851485a2eec1
2020-12-16 10:47:46 +00:00
Jennifer Johnson
adb2c83e09 cmd/uplink: adds register, url, and dns flags to uplink share
and replaces access grant with access

uplink share <path> --> creates access grant

uplink share --register <path> --> registers access grant

uplink share --url <path> --> creates URL, implies register and public

uplink share --dns <hostname> <path> --> creates dns info, implies register and public

Change-Id: I7930c4973a602d3d721ec6f77170f90957dad8c0
2020-12-14 20:51:44 -05:00
Stefan Benten
3c77825b19 cmd: move ca-certificates build step to be arch independent
It turns out, that running a docker image build for specific
arches is not possible from amd64 (eg. installing ca-certificates).

Change-Id: I8b8f002b7e532fb4a0c6542d5b573c294c501068
2020-12-08 21:49:41 +01:00
Stefan Benten
a71c908c1e
cmd: add ca-certificates to Docker images (#3986) 2020-12-08 01:38:33 +01:00
Bill Thorp
47e008b714 cmd/uplink: Write to AWS Credentials file more safely from uplink access register
Jeff provided feedback on https://review.dev.storj.io/c/storj/storj/+/3176 after
the changeset was already merged.  I attempt to address that feedback here.

Change-Id: Ibc7dba3e4e2c73736042fe4b4ee49ce679ba7f44
2020-12-01 17:47:14 +00:00
Bill Thorp
65919f9f7d cmd/uplink: add --aws-profile flag to uplink access register
Added flag to append a new profile to ~/.aws/credentials using
the provided profile name.  This is handy for the AWS CLI, so
you can do things like 'aws configure get aws_access_key_id --profile=me'

Change-Id: I0469a18ca76e078624ed455a06bd7aabd95a1b97
2020-11-30 09:39:01 -05:00
Jeff Wendling
54ae9b040e cmd/uplink: allow public access registration
Change-Id: I1316f10158d8a466bc80d6548ad0773016bfb916
2020-11-18 17:40:30 -05:00
Bill Thorp
5fe3d2dea7 cmd/uplink Allow use of named accesses in uplink register
Previously uplink register only accepted a fully serialized access grant.
This is kind of annoying, I changed it so that it could also use access names.

Change-Id: If6d4d1baa8d4fb3d87fdedb895d459fa12743f1a
2020-11-18 12:23:57 -05:00
Isaac Hess
e32466696a cmd/uplink: add access register command
This command allows you to register your access with a separate service
(hosted gateway).

Change-Id: I67200cd4553fd41e9b1a56cca7ccc6553136a76a
2020-11-10 15:25:57 -07:00
Egon Elbre
2268cc1df3 all: fix linter complaints
Change-Id: Ia01404dbb6bdd19a146fa10ff7302e08f87a8c95
2020-10-13 15:59:01 +03:00
Monty Anderson
7710e083d4
cmd/uplink: update CLI format (#3773) 2020-09-23 12:54:00 +02:00
Yingrong Zhao
af773ec8a6 cmd/uplink: use DeleteBucketWithObjects for bucket force deletion
This PR updates `uplink rb --force` command to use the new libuplink API
`DeleteBucketWithObjects`.
It also updates `DeleteBucket` endpoint to return a specific error
message when a given bucket has concurrent writes while being deleted.

Change-Id: Ic9593d55b0c27b26cd8966dd1bc8cd1e02a6666e
2020-09-02 16:39:20 +00:00
Egon Elbre
94a09ce20b all: add missing dots
Change-Id: I93b86c9fb3398c5d3c9121b8859dad1c615fa23a
2020-08-11 17:50:01 +03:00
Egon Elbre
080ba47a06 all: fix dots
Change-Id: I6a419c62700c568254ff67ae5b73efed2fc98aa2
2020-07-16 14:58:28 +00:00
Egon Elbre
29ccda5d47 cmd/uplink/cmd: remove spf13/cast dependency
Change-Id: I8013014b0b37966128e673155d851705ee1d43db
2020-07-13 11:10:48 +00:00
Egon Elbre
c40d5043e1 cmd/uplink: pass useragent flag to library
Change-Id: I8cf12c978c01eca968086560be0c28809eb2ba7d
2020-07-03 16:41:35 +03:00
Isaac Hess
cad21f11e5 cmd/uplink: Add revoke command
Change-Id: I8ae74da5c9171429b970d76598793a3227ed4d11
2020-06-22 13:50:07 -06:00
JT Olio
2bd9067ad5 cmd/uplink: tighter linksharing security
when a user runs `uplink share`, they get a bunch of results back,
given their configuration and existing access. one of the results
is a URL for in-browser sharing and hosting of the file.

first off, we want to make sure this URL is read only. we want to
avoid a situation where someone post this URL to some public
location, not realizing the access allows writes or deletes. if
a user really wants a URL with write/delete access, they can
construct it themselves.

secondly, we want to make sure the url is sharing a single path or
path prefix. having a url for multiple paths/path prefixes
indepedently again can be constructed of course, but should not
be the default behavior

Change-Id: I2ca2ebeea9f1c7d4bfbd7a437a32dc7a3b2a32cc
2020-06-16 15:02:53 -06:00
Egon Elbre
34db4a80fd ci: fix staticcheck failures
Change-Id: I176fb24214755a1940a0a1a4e9cc8e39f184870b
2020-06-05 13:15:34 +00:00
Michal Niewrzal
056ae7ffa8 scripts/test/rollingupgrade: set correct satellite address for imported
access

Addition: use always latest gateway release

Change-Id: I5e5231e6da4b6f7900cb71bb6e227901474270ea
2020-06-04 15:04:10 +00:00
Michal Niewrzal
79a562c3ed cmd/uplink/cmd: improve uplink remove bucket command with force flag
* don't stop on single delete failure
* display number of successful and unsuccessful deletes

Change-Id: Id7f29055f250aad515e46622c2bfd3596055b93a
2020-06-01 07:56:14 +00:00
Michal Niewrzal
84892631c8 private/testplanet: remove old libuplink from testplanet
Change-Id: Ib1553f84d0b3ae12a5b00382f0f53357b6a273e2
2020-05-28 13:50:23 +00:00
Michal Niewrzal
a4c19b3359 cmd/uplink/cmd: remove port detection when selecting satellite while
setup

We don't have default ports as a part of configuration anymore because
satellite-addr flag was removed.

Change-Id: Ibf9fc4b399beaf51ebb9461de2d8994a322f9686
2020-05-26 13:25:51 +02:00
Michal Niewrzal
340700eda9 cmd/uplink/cmd: remove old libuplink from Uplink CLI code
Change-Id: I38444a67e3bdec5f63c56895b5fc362464e7dce1
2020-05-22 09:08:58 +00:00
Bill Thorp
bcb867f1de cmd/uplink: add UserAgent flag for partner value attribution
This changeset allows a user agent string to be set during uplink setup, which
is thereafter used for partner value attribution.  EG

uplink setup --client.user-agent ”MyCompany”

Change-Id: Iefa8755fccc06acb8a303a342b943cece44a81f7
2020-05-22 07:39:14 +00:00
Jeff Wendling
f93d764998 cmd/uplink: properly path escape shares
Fixes #3829

Change-Id: I0f900a392239fcc064cea968fc7a5bc373ee26a1
2020-05-04 17:00:25 +00:00
Yingrong Zhao
0b80d60491 cmd/uplink: disable tlemetry if user opt out
Change-Id: I793e6428d04e4f422d1f8292e99b233c70d9dd6e
2020-05-04 14:46:49 +00:00
Yingrong Zhao
c5309a3f91 cmd/uplink: set sample rate for tracing to be 1 when tracing is enabled
when tracing is enabled, we should also set sampling rate to
a non-zero value. For now, we will set it to 1.
Uplink CLI users should be able to override it with the sample
flag.

Change-Id: I8bcf514fb14c2a1c4349b7957dd24ec23e4a85e5
2020-04-28 20:15:28 +00:00
Yingrong Zhao
9b4a3f8fcc cmd/uplink: use tracing.enabled flag
Previously we are using tracing.sampled to be the switch for turning on/off tracing.
However we would like to separate sampling rate from being the switch,
so we can set sampling rate to be 0 but still intialize tracing for
satellite and storagenodes

Change-Id: I27e6ba25ea6f6b612b4e1a57cf1301889ded41ec
2020-04-27 17:54:57 +00:00