Commit Graph

17 Commits

Author SHA1 Message Date
Jeff Wendling
4426470173 cmd/uplinkng: rename parallelism flags, small fixes
this makes the flags match rclone nomenclature

fixes test-uplinkng to use the temporary config dir
instead of the machine default, and clean up some.

bumps clingy so that the command errors when an unknown
command is specified.

also fixes some printfs in share to use clingy stdout.
it still does some external actions that should be
passed through a ulext.External for mocking, but
that's ok for now.

Change-Id: Icc231e7e26393541c312396fec907b640b97718e
2021-12-22 13:13:30 +00:00
Jeff Wendling
9615bd191f cmd/uplinkng: update clingy for better boolean support
this allows commands like

    uplinkng cp -r sj://foo sj://bar

to work correctly, rather than complain that sj://foo is
not a boolean.

Change-Id: I003e47aabb85566bc2b454851cf55043b17ee7ea
2021-12-14 09:41:38 +00:00
Jeff Wendling
c8c4e8835e cmd/uplinkng: file level parallel copy
Change-Id: I074abca67ea6f4a2fd0983fe452c74f76ee62c6a
2021-12-13 12:14:39 -05:00
Jeff Wendling
baaa96c208 cmd/uplinkng: introduce MultiWriteHandle
Change-Id: I6acf93141ddfa62728164818a322120ed6956b00
2021-12-10 10:23:04 -05:00
Jeff Wendling
34890c9195 cmd/uplinkng: introduce MultiReadHandle
Change-Id: I57b98b5e1406e7b38edf3bc65907d9796a1a663b
2021-12-10 09:30:25 +00:00
Michał Niewrzał
24cf7e8ea6 cmd/uplinkng: add mv command
Add ability to move files and objects.

Change-Id: I4929da730984c06aa578678b1d8c8e9b4aceade8
2021-11-22 09:07:24 +00:00
Clement Sam
16a334020f cmd/uplinkng: add ranged download
This change adds the ability to download byte ranges
to uplinkng.

Extended the uplinkng Filesystem interface with Stat
method and an OpenOptions struct as parameter for the
Open method.

Also added a few tests for the ranged download

Change-Id: I89a7276a75c51a4b22d7a450f15b3eb18ba838d4
2021-11-03 19:51:25 +00:00
Jeff Wendling
987cb6ab11 cmd/uplinkng: allow removing pending objects
Change-Id: I12457e7d6fb28492ed4b6c5816f78aa7820fed6b
2021-10-20 12:25:25 -04:00
Jeff Wendling
59b8422318 cmd/uplinkng: implement revoke and normalize some language
this makes the distinction between an "access name" and
an "access value" and talks about which is expected for
commands. most are "access name or value".

Change-Id: I43c0043a17d37e89ab5f87388ae9e890a8b59958
2021-08-11 02:30:13 +00:00
Jeff Wendling
08d860570b cmd/uplinkng: parallelsm and a ton of fixes
this was just supposed to add parallel uploads/downloads
and it does do that, but i then found a bunch of bugs
with respect to path handling that i thought i had under
control. oops.

so this adds a ton of tests and tries to make the logic
in ulloc to be more consistent. almost all of the actual
file handling bits and knowledge happens in cmd_cp now
where it should belong.

additionally, the s3 command has the behavior that if your
bucket has the file s3://bucket/file, then executing
s3 ls s3://bucket/fi returns nothing. this change makes
uplinkng match that behavior even if i don't personally
like it.

a big portion of the weirdness is the concept introduced
that i've named "directoryish", which intends to capture
the behavior that if a user copies a file to that location
then the base name of the source should be appended on
rather than a direct copy. this concept is entirely a
based on the string value and not the actual filesystem
state. hence, the cp command is responsible for checking
if local paths are actually a directory, and adding a
trailing slash if necessary to make them "directoryish".
additionally, the empty key for a bucket and the empty
string for local paths are considered "directoryish".

Change-Id: I9120d18616fd813b29ff81beed4f5993caa99fb6
2021-08-11 02:30:06 +00:00
Jeff Wendling
ef7b89cc03 cmd/uplinkng: remove global flags
this changes globalFlags to be a ulext.External
interface value that is passed to each command.

rather than have the ulext.External have a Setup
call in the way that the projectProvider used to
we make all of the state arguments to the functions
and have the commands call setup themselves.

the reason it is in its own package is so that
cmd/uplinkng can import cmd/uplinkng/ultest
but cmd/uplinkng/ultest needs to refer to whatever
the interface type is to call the function that
creates the commands.

there's also quite a bit of shuffling around of
code and names. sorry if that makes it tricky
to review. there should be no logic changes, though.

a side benefit is there's no longer a need to do
a type assertion in ultest to make it set the
fake filesystem to use. that can be passed in
directly now. additionally, this makes the
access commands much easier to test.

Change-Id: I29cf6a2144248a58b7a605a7ae0a5ada5cfd57b6
2021-07-06 17:26:51 -04:00
Jeff Wendling
f474bb6179 cmd/uplinkng: update for breaking clingy changes
clingy changed some and this is just fixes for that

Change-Id: I729aed6329fe0988fcb9b4407f16966a753b3204
2021-07-06 17:26:51 -04:00
Jeff Wendling
ce87652a8c cmd/uplinkng: rm
this implements the rm command which has to add
a Remove method to the fileystem interface and
implement it for local, remote and test filesystems.

Change-Id: Id41add28f01938893530aae0b4b73c8954e9b715
2021-06-14 15:23:41 -04:00
Jeff Wendling
d73287f043 cmd/uplinkng: tests for cp
this adds some stuff to ultest so that the set of
files created by a test can be inspected after so
that we can write some tests for the cp command
to observe that it does what it is supposed to do.

Change-Id: I98b8fb214058140dfbb117baa7acea6a2cc340e1
2021-06-14 15:23:41 -04:00
Jeff Wendling
98be54b9a3 cmd/uplinkng: refactor into some focused packages
the directory was starting to get pretty large and
it was making it hard to pick concise names for
types and variables. this moves the location
stuff into a cmd/uplinkng/ulloc package, the
filesystem stuff into a cmd/uplinkng/ulfs package,
and the testing stuff into a cmd/uplinkng/ultest
package.

this should make the remaining stuff in cmd/uplinkng
only the business logic of how to implement the
commands, rather than also including a bunch of
helper utilities and scaffolding.

Change-Id: Id0901625ebfff9b1cf2dae52366aceb3b6c8f5b6
2021-06-14 15:23:41 -04:00
Jeff Wendling
46a3242ed4 cmd/uplinkng: cp
Change-Id: I9c251028d9f72572f4d42815de31d44517cd5525
2021-06-02 14:35:27 +00:00
Jeff Wendling
a1bf9ab6de cmd/uplinkng: initial commit with skeleton
Change-Id: I764618cc60c46882955e9b08b72b3c162aa4929f
2021-05-18 10:04:44 -04:00