This change adds the ability to download byte ranges
to uplinkng.
Extended the uplinkng Filesystem interface with Stat
method and an OpenOptions struct as parameter for the
Open method.
Also added a few tests for the ranged download
Change-Id: I89a7276a75c51a4b22d7a450f15b3eb18ba838d4
this makes the distinction between an "access name" and
an "access value" and talks about which is expected for
commands. most are "access name or value".
Change-Id: I43c0043a17d37e89ab5f87388ae9e890a8b59958
this was just supposed to add parallel uploads/downloads
and it does do that, but i then found a bunch of bugs
with respect to path handling that i thought i had under
control. oops.
so this adds a ton of tests and tries to make the logic
in ulloc to be more consistent. almost all of the actual
file handling bits and knowledge happens in cmd_cp now
where it should belong.
additionally, the s3 command has the behavior that if your
bucket has the file s3://bucket/file, then executing
s3 ls s3://bucket/fi returns nothing. this change makes
uplinkng match that behavior even if i don't personally
like it.
a big portion of the weirdness is the concept introduced
that i've named "directoryish", which intends to capture
the behavior that if a user copies a file to that location
then the base name of the source should be appended on
rather than a direct copy. this concept is entirely a
based on the string value and not the actual filesystem
state. hence, the cp command is responsible for checking
if local paths are actually a directory, and adding a
trailing slash if necessary to make them "directoryish".
additionally, the empty key for a bucket and the empty
string for local paths are considered "directoryish".
Change-Id: I9120d18616fd813b29ff81beed4f5993caa99fb6
this changes globalFlags to be a ulext.External
interface value that is passed to each command.
rather than have the ulext.External have a Setup
call in the way that the projectProvider used to
we make all of the state arguments to the functions
and have the commands call setup themselves.
the reason it is in its own package is so that
cmd/uplinkng can import cmd/uplinkng/ultest
but cmd/uplinkng/ultest needs to refer to whatever
the interface type is to call the function that
creates the commands.
there's also quite a bit of shuffling around of
code and names. sorry if that makes it tricky
to review. there should be no logic changes, though.
a side benefit is there's no longer a need to do
a type assertion in ultest to make it set the
fake filesystem to use. that can be passed in
directly now. additionally, this makes the
access commands much easier to test.
Change-Id: I29cf6a2144248a58b7a605a7ae0a5ada5cfd57b6
this implements the rm command which has to add
a Remove method to the fileystem interface and
implement it for local, remote and test filesystems.
Change-Id: Id41add28f01938893530aae0b4b73c8954e9b715
this adds some stuff to ultest so that the set of
files created by a test can be inspected after so
that we can write some tests for the cp command
to observe that it does what it is supposed to do.
Change-Id: I98b8fb214058140dfbb117baa7acea6a2cc340e1
the directory was starting to get pretty large and
it was making it hard to pick concise names for
types and variables. this moves the location
stuff into a cmd/uplinkng/ulloc package, the
filesystem stuff into a cmd/uplinkng/ulfs package,
and the testing stuff into a cmd/uplinkng/ultest
package.
this should make the remaining stuff in cmd/uplinkng
only the business logic of how to implement the
commands, rather than also including a bunch of
helper utilities and scaffolding.
Change-Id: Id0901625ebfff9b1cf2dae52366aceb3b6c8f5b6
