The API generator was generating invalid code when types were defined in
a main package because the generated Go code was defining in import from
it.
This commit update the Go generator to panic with a explicit error
message if that situation happens.
The commit also add a new endpoint to the example with a named types
(i.e. no anonymous) to show that the Generator works fine with them.
Change-Id: Ieddd89c67048de50516f7ac7787d602660dc4a54
The API generator didn't generate valid TypeScript code when using
Go anonymous types.
This commit fixes that issue creating names for anonymous types.
Change-Id: Ice0748d8650686e3d3979523b8f218dc20eade5a
Add a few validations to panic with a nicer message or abort rather than
generating invalid code.
Also improve the panic message wrapping a standard error with errs2 at
the time that it's returned.
Change-Id: I1393933eb5f0bc3f86646bf4d0acfc64626efbe0
Add a test for Types.All method because we may need to adjust the logic
in future commits and we want to detect what has changed for good or
bad.
Change-Id: I1db4bf67db3c87513cb9aeb7b942c6c132fc4dd1
Detects whether or not TCP fastopen is supported by running sysctl to
grab the current value of net.inet.tcp.fastopen.server_enable.
If not enabled, directs the operator to use sysctl to set it temporarily
and /etc/sysctl.conf to set it on-boot.
Setting the socket option always works whether it is enabled or not.
Verified that fastopen works on FreeBSD 13. Did not attempt on earlier
versions but support has been there since FreeBSD 10.
Change-Id: I2e0c457558a6fa7b7a1b18bc3c6684aff50b81a2
Implements tryEnableFastOpen by creating a localhost socket and enabling
the TCP_FASTOPEN socket option. On builds where TCP_FASTOPEN isn't
available, setting the socket option fails and tryEnableFastOpen returns
false.
This was verified on a supporting build (latest Windows 10) and an
unsupporting build (Windows 8.1, couldn't find an ISO for an older
Windows 10 build).
Change-Id: I497117dc2f04acdd2b0cc836e20d12d69076b939
as GetParticipatingNodes and GetNodes, respectively.
We now want these functions to include offline and suspended nodes as
well, so that we can force immediate repair when pieces are out of
placement or in excluded countries. With that change, the old names no
longer made sense.
Change-Id: Icbcbad43dbde0ca8cbc80a4d17a896bb89b078b7
The old way did not properly handle escaping, e.g. if the value of a
query param contained `&` or `=` inside it. By using
url.searchParams.set, we can safely add these types of arguments to the
path.
Change-Id: I62d3883b14f9d5a517e4a3d58f019014b46fd1b4
Previously the base path for the API was hardcoded to `/api` and the
specified version.
This was not obvious that the generated code was setting that base path
and it was not flexible for serving the API under a different path than
`/api`.
We will likely need to set a different base path if we pretend to serve
the new back office API that we are going to implement alongside the
current admin API until the new back office is fully implemented and
verified that works properly.
This commit also fix add the base path of the endpoints to the
documentation because it was even more confusing for somebody that wants
to use the API having to find out them through looking to the generated
code.
Change-Id: I6efab6b6f3d295129d6f42f7fbba8c2dc19725f4
Generate the documentation and the typescript code for the example of
that we have to generate an API through the API generator.
The JSON Go struct field tags are needed because the typescript
generator require them, otherwise it panics.
The test had to be adjusted according to match the tags so Go consider
the structs the same type, otherwise, it doesn't compile.
Change-Id: I3e4ebff9174885c50ca2058b86b7ec60e025945c
Make the link more human-friendly - to contain the text of the group and
endpoint names.
Also link back to list of endpoints from each endpoint.
Change-Id: Ia3e2ebe20b58b5f60ecefe9d35fb8fd90dd4b4d7
This change makes it easier for someone reading the documentation to see
a full list of supported endpoints, and have direct links to the
details.
Change-Id: I46e2f809cfa2760845898eaa3d99db9066d435ef
This change adds an endpoint to get a user's projects, similar to
the OwnedProjects GraphQL query.
The console.ProjectInfo struct has been renamed to UpsertProjectInfo
to more accurately reflect its use.
Issue: https://github.com/storj/storj/issues/6135
Change-Id: I802fe4694a5cc75a9df2b565476f6e6f473431d4
This change adds request IDs to requests, logs them as part of audit
logs and sends to the client on error. This is to improve debugging
of customer issues.
Issue: https://github.com/storj/storj/issues/5898
Change-Id: I801514b547d28d810552d91aa7c8502051e552bf
This change creates the ability to run a server separate from the
console web server to serve the front end app. You can run it with
`satellite run ui`. Since there are now potentially two servers instead
of one, the UI server has the option to act as a reverse proxy to the
api server for local development by setting `--console.address` to the
console backend address and `--console.backend-reverse-proxy` to the
console backend's http url. Also, a feature flag has been implemented
on the api server to retain the ability to serve the front end app. It
is toggled with `--console.frontend-enable`.
github issue: https://github.com/storj/storj/issues/5843
Change-Id: I0d30451a20636e3184110dbe28c8a2a8a9505804
Having separate process/pod only for sending bloom filters once a week
is a bit waste. After reconfiguring production settings to use sender in
core we can remove also GC sender peer code.
https://github.com/storj/storj/issues/5979
Change-Id: I6efe3ec073f96545e1f70ad13843f8ccdf923ee8
placement.AllowedCountry is the old way to specify placement, with the new approach we can use a more generic (dynamic method), which can check full node information instead of just the country code.
The 90% of this patch is just search and replace:
* we need to use NodeFilters instead of placement.AllowedCountry
* which means, we need an initialized PlacementRules available everywhere
* which means we need to configure the placement rules
The remaining 10% is the placement.go, where we introduced a new type of configuration (lightweight expression language) to define any kind of placement without code change.
Change-Id: Ie644b0b1840871b0e6bbcf80c6b50a947503d7df
There are many case where the keywords `free` and `available`
are confused in their usage.
For most cases, `free` space is the amount of free space left
on the whole disk, and not just in allocation while
`available` space is the amount of free space left in the
allocated disk space.
What the user/sno wants to see is not the free space but the
available space. To the SNO, free space is the free space
left in the allocated disk space.
Because of this confusion, the multinode dashboard displays
the `free` disk space instead of the free space in the
allocated disk space https://github.com/storj/storj/issues/5248
While the storagenode dashboard shows the correct free space
in the allocation.
This change fixes the wrong free disk space. I also added a
few comments to make a distinction between the `free`
and `available` fields in the `DiskSpace*` structs.
Change-Id: I11b372ca53a5ac05dc3f79834c18f85ebec11855
Currently we are using Reliable to get missing pieces for repair
checker. The issue is that now checker is looking at more things than
just missing pieces (clumped/off, placement pieces) and using only node
ID is not enough. We have issue where we are skipping offline nodes from
clumped and off placement pieces check.
Reliable was refactored to get data (e.g. country, lastNet) about all
reliable nodes. List is split into online and offline. This data will be
cached for quick use by repair checker. It will be also possible to
check nodes metadata like country code or lastNet.
We are also slowly moving `RepairExcludedCountryCodes` config from
overlay to repair which makes more sens for it.
This this first part of changes.
https://github.com/storj/storj/issues/5998
Change-Id: If534342488c0e440affc2894a8fbda6507b8959d
Add some code to generate a basic markdown file documenting a generated
API. Generate this document for the API in
satellite/console/consoleweb/consoleapi/gen.
The documentation is not completely correct, as it may include some
values in the request body that are not actually usable by the
requester. This can be fixed by making sure all types used within the
generated API are properly annotated with `json` tags.
Issue: https://github.com/storj/storj-private/issues/244
Change-Id: I57b259967fb0db8f548b6598a10c825da15ba723
this is a very old tool built in the very early days
of v3, when we didn't know how the network would be
used. this tool anticipated being able to query remote
nodes for internal state. we don't do that. i don't
think anyone uses this.
Change-Id: Ie1ded3ecbedb09313f2d6fc721039e0f15e4ee85
The query for GetNodesNetworkInOrder is causing far too much load on the
database. Since it is not critical that the repair checker have
perfectly up-to-date node network information, we can use a cache
instead.
Change-Id: I07ad45bfdeb46529da093941a06c2da8a00ce878
By implementing a simpler Range, we can get rid of many methods
defined for storage.KeyValueStore.
Initially KeyValueStore was used to manage objects, hence it required
much more complex behaviour. However, we've moved away from this
implementation, hence we can significantly simplify everything.
Change-Id: Iaaa8c9c83d8601ad3c894767f17c0f5b356f7253
The blobstore implementation is entirely related to storagenode, so the
rightful place is together with the storagenode implementation.
Fixes https://github.com/storj/storj/issues/5754
Change-Id: Ie6637b0262cf37af6c3e558556c7604d9dc3613d
With this change we are replacing parsing code with existing go-redis
util.
We also switch redis client to version 9.
Change-Id: Ie4a651e3ae6960e68958c690873925d319b70e10
This flag was in general one time switch to enable versions internally.
New we can remove it as it makes code more complex.
Change-Id: I740b6e8fae80d5fac51d9425793b02678357490e
to support TCP_FAST_OPEN, we're considering just using
two TCP connections in parallel per request, one with
and one without. this allows us to safely fire both
concurrently without stressing out the node too much.
see https://review.dev.storj.io/c/storj/storj/+/9933
Change-Id: I9aa8a0252350db5ace04ee125bfe469203e980ec
The code responsible for generating TypeScript classes has been
separated from the rest of the TypeScript generation code so that other
packages may take advantage of this functionality.
References #5494
Change-Id: I97eabd430bd6a5f748eafaf8b1d783977e75e660
