Commit Graph

66 Commits

Author SHA1 Message Date
Ivan Fraixedes
6885ca27fb private/apigen: Inject middleware code after param parsing
Inject the middleware code after the query and path parameters parsing
because in the case that a middleware needs to execute a more expensive
logic, it isn't executed if there are malformed or missing parameters.

Change-Id: I883ade8ee42d25accb153c78d9f583d883730d6a
2023-11-27 23:02:44 +00:00
Ivan Fraixedes
269dd5602e private/apigen: Don't print empty description & version in docs
If the API description or version is empty, don't print out in the docs
a placeholder for them.

In commit bcde51138d a space between the
placeholder and description was removed because it was printing a
trailing space if the description was empty, but that introduced another
bug, which is not adding the space when the description is empty.

This commit fixes the bug, no printing the placeholder if the
description is empty, otherwise print it with the corresponding blank
space in between the placeholder and the description.

Change-Id: I19c92c3dd7dfb38b8b2a43821bafc726ffad59d7
2023-11-27 16:53:33 +00:00
Ivan Fraixedes
adcd810e37 private/apigen: Allow to customize handlers logic
The API generator doesn't have a way to customize each Go handler
endpoint unless that the Go generator is modified.

This commit adds a way to customize each endpoint injecting instances of
types that implement an interface (Middleware) that return the code to inject.

To show how it works, the commit get rid of the 2 fields that we used to
customize the authentication request with the logic that the
satellite/console/consoleweb/consoleapi needs and replace the hardcoded
customization using this new way to customize handlers.

This new way should allow to hook the satellite/admin/back-office
authorization into the handlers using a Middleware implementation.

Change-Id: I894aa0026b30fa2f4a5604a6c34c22e0ed582e2b
2023-11-23 06:57:40 +00:00
Jeremy Wharton
587fa8fdff private/apigen: use correct TS type for nillable fields
This change gives the proper type to TS class fields generated from
nillable Go struct fields. Previously, Go struct fields having a nil
representation ([]Type, *Type, etc.) were translated into TypeScript as
"Type | undefined". This isn't correct because these fields, when nil,
are given the value "null" when marshalled into JSON. This change fixes
this issue by giving these fields the type "Type | null".

Change-Id: I5a1a83eb3810a3cba10895bb2f0f75ca5fd7d1b5
2023-11-21 17:58:07 +00:00
Jeremy Wharton
0c591fa25a private/apigen: handle omitempty JSON option in structs
This change makes the TypeScript API code generator properly handle
struct fields with the "omitempty" option in the JSON struct tag.

Change-Id: I9b22ce33a8b8c39c115ec827a8e5b7e85d856f83
2023-11-18 07:09:42 -06:00
Ivan Fraixedes
bcde51138d private/apigen: Remove trailing space in doc generator
Remove a trailing space that the doc generator was adding.

Change-Id: I5c24478f79726afbaa5c6772d569b35cafe7c288
2023-11-17 11:44:50 +00:00
Ivan Fraixedes
e67691d51c private/apigen: Remove support for anonymous types
I made several commits to add support to the API generator for
anonymous types because the example was using them.

Later, I was told that we don't need to support them, so the example
should have never used anonymous types.

Supporting anonymous types has added complexity to the API generator and
we are finding bugs in corner cases whose fixes lead to adding more
complexity and some fixes doesn't seem to have a neat solution without
doing a big refactoring.

We decided to reduce the side effects that supporting anonymous type has
brought just removing it.

This commit remove the support for anonymous types and reflect that in
the example.

Change-Id: I2f8c87a0db0e229971ab1bef46cca16fee924191
2023-11-16 12:29:21 +00:00
Jeremy Wharton
032faefa4b private/apigen: fix URL construction in generated TypeScript code
This change fixes an incorrect invocation of the URL object constructor
in generated TypeScript HTTP clients.

Change-Id: I9011bc535f2096374d20b74b401d4cc38a0451fb
2023-11-16 01:12:58 +00:00
Jeremy Wharton
a52934ef4d private/apigen: prevent self imports
This change prevents Go code produced by the API generator from
importing its own package. Previously, we tried to prevent self imports
by skipping import paths whose last segment matched the generated Go
code's package name. However, aliased imports circumvented this.
We now require API definitions to define the Go package path so that we
can compare this with the import path directly.

Change-Id: I7ae7ec5e1a342d2f76cd28ff72d4cd7285c2820a
2023-11-15 21:41:20 +00:00
Jeremy Wharton
f1ce0ab6d0 private/apigen/example: use hardcoded time
This change makes the example API definition use a hardcoded timestamp
rather than the current time so that files aren't unnecessarily
changed each time the API code is generated.

Change-Id: I72e4ec98d29345b9f16ca2ca38a91e593f83ea87
2023-11-15 16:52:18 +00:00
Ivan Fraixedes
48877c05cc
private/apigen: Fix TS generator to align with linter
I introduced some subtle linter issues when I added the APIError class
and added the TypeScript mock generator.

This commit addresses them, so the linter doesn't yell about the
TypeScript generated sources.

Change-Id: Icc7dfa4169a228b1a5144d4a292f4350ee5ef9f0
2023-11-03 16:41:54 +01:00
Ivan Fraixedes
74757ffc1d private/apigen/example: Fix client TS generated file
We introduced some changes in 882c9d64e4
and one reviewer spot an issue which was fixed, but I forgot to
regenerate the files and the generated Typescript client got stale.

This commit regenerate the example code, so the generated code is align
with the current generator version.

Change-Id: Idc023adc9c6e230578f9e77e492126355b006203
2023-11-02 14:24:30 +00:00
Ivan Fraixedes
9338f3f088
private/apigen: Add a TypeScript client mock generator
Implement a TypeScript client mock generator to generate mocks with
static data specified in the API definition.

Change-Id: I11419f4bbf72576fcd829f9d4acd8471034ca571
2023-10-25 14:43:01 +02:00
Ivan Fraixedes
882c9d64e4
private/apigen: Create class for API errors (TypeScript)
Create a class that inherits the Javascript Error class to use it when
throwing errors on the TypeScript client when the ok property of the
Fetch API Response object is false, which means that the server
responded with a status code less than 200 or greater than 299.

The APIError class contains apart of the message, the response status
code which may be useful for the caller to decide what to do, for
example, it may decide to retry the call on certain response status
errors.

Change-Id: Ic48466b40cbf134a27d5c92a4af9f93232e84fca
2023-10-25 14:43:00 +02:00
Ivan Fraixedes
3e71ea555d private/apigen: Remove stale TODOs & improve doc
Remove some stale TODO comments and make a method documentation more
specific.

Change-Id: I21b9df0f22a09698014518b5a675eb115d1ba4a4
2023-10-19 12:42:58 +00:00
Jeremy Wharton
6d03b92ea6 private/apigen: fix TS class generation from collection of named struct
This change fixes an issue where the TypeScript API code generator
would silently fail to generate class definitions for named structs
used as the element type of array or slice struct fields.

This issue caused invalid TypeScript code to be generated from the
frontend config struct because it contained a slice of partnered
satellite structs. The TypeScript class definition of the frontend
config referenced the nonexistent partnered satellite class.
The satellite frontend config uses, so this issue caused the

Change-Id: Idfcb6ec1bbc603a43033ee4afb5b421b7454855c
2023-10-17 22:45:07 -05:00
Ivan Fraixedes
386a978310 private/apigen: Validate param types
The Go generator only supports certain types as query and path
parameters and it panics on any an unsupported type.

The Document and TypeScript generator don't have any validation for
them. TypeScript generator generates code that compiles, however, it
won't work properly with all the types not supported by the Go
generator.

Because it doesn't make sense that some types may work on the TypeScript
generator, while the Go generator doesn't, doing the validation in the
Param constructor is better because it reports the issue without having
to run the Go generator and it gives a more understanding panic message.

TypeScript generator generates code that works properly with all the
types supported by the Go generator, hence, there isn't any change int
he TypeScript generator in this commit.

Change-Id: I03085283942a98341726a1560f511a46540df9f5
2023-10-10 12:37:20 +00:00
Ivan Fraixedes
99c4359062 private/apigen: Avoid clashes of types names
The TypeScript generator create types of anonymous structs. When those
anonymous structs are in endpoints with the same name, but in different
endpoint groups, the generator panic because it create different types
with the same names.

This commit fixes that problem through using the endpoint group prefix
to create the types with different names.

Change-Id: Ibe87532609ce824b80951326f9777ed5b0cc2f7a
2023-10-05 07:30:09 +00:00
Ivan Fraixedes
3193ff9155 private/apigen: Don't force casing for API group name/prefix
The API generators rely on the Name and Prefix fields of the
EndpointGroup type to generate code.

Conventional naming code requires using upper or lower case for types,
functions, etc, however requiring the user to set this fields with the
correct casing seems cumbersome for them because they can be adjusted
depending where those values are used on the generated code.

This commit lifts the restriction for the user and adjust the casing of
them according to where they are used.

Change-Id: I700a879d13b4789b4d6ba0519b4d7508061eac73
2023-10-05 06:48:50 +00:00
Ivan Fraixedes
47f344927d
private/apigen: Fix TS generator to apply naming conventions
Fix the TypeScript generator to generate code using the common
TypeScript conventions.

Closes https://github.com/storj/storj/issues/6360

Change-Id: I244896feea389670eca0df95d3ac58e25d600e14
2023-10-04 13:47:33 +02:00
Ivan Fraixedes
71c9547de5
private/apigen: Don't allow certain types for Request/Response
We cannot map certain types to TypeScript, hence we verify them in the
Endpoint.Validate method.

Even some types could be somehow mapped, we don't want to add more
complexity or allow types that don't make sense to be for a request or
response.

Change-Id: I51ecee286e637b1160e967d77f9ce6c7403ddfdc
2023-10-04 13:46:54 +02:00
Ivan Fraixedes
956109a097
private/apigen: Rename Endpoint fields
Rename the Endpoint fields MethodName and RequestName because they were
confusing for what they are used.

This commit also adds some validations for these fields values and other
validations for Endpoint and EndpointGroup to avoid generating invalid
code.

It also include some tests for these new validations.

Closes https://github.com/storj/storj/issues/6333

Change-Id: Iaabfc33935517889e3729c8b37be51a55eea366c
2023-10-04 13:46:54 +02:00
Ivan Fraixedes
ac86eb397a
private/apigen: Fix bug TypeScript optional class members
The TypesScript generator generates classes and checks for Go struct
field if the struct type is of a nullable type, however, it should check
if the field types is nullable because it doesn't make sense to check
that struct field for each field nor to assign to the field if it's
nullable or not depending on it.

Change-Id: Ia22a609a17752f520233c006cba17685fe142b32
2023-10-04 13:46:08 +02:00
Ivan Fraixedes
9d7ef17a26
private/apigen: Fix code generation for slices & arrays
Fix the API generator to generate valid TypeScript code when using
slices an arrays of any type (base types, struct types, anonymous struct
types, etc.).

Closes https://github.com/storj/storj/issues/6323

Change-Id: I580ae5305c58f65c2e4f4a35d14ca4ee509a9250
2023-10-02 17:05:35 +02:00
Ivan Fraixedes
48d7be7eab
private/apigen: Panic types defined in main package
The API generator was generating invalid code when types were defined in
a main package because the generated Go code was defining in import from
it.

This commit update the Go generator to panic with a explicit error
message if that situation happens.

The commit also add a new endpoint to the example with a named types
(i.e. no anonymous) to show that the Generator works fine with them.

Change-Id: Ieddd89c67048de50516f7ac7787d602660dc4a54
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
7ab7ac49c8
private/apigen: Remove TypeScript errors from generated code
Remove the "duplicate identifier: 'path'" TypeScript errors from the
generated code.

Change-Id: I6b411a5cee720e2a16353034627954616c480f8a
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
00484429d6
private/apigen: Generate valid TypeScript code with anonymous types
The API generator didn't generate valid TypeScript code when using
Go anonymous types.

This commit fixes that issue creating names for anonymous types.

Change-Id: Ice0748d8650686e3d3979523b8f218dc20eade5a
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
a9901cc7d0
private/apigen: Add validations to improve usage
Add a few validations to panic with a nicer message or abort rather than
generating invalid code.

Also improve the panic message wrapping a standard error with errs2 at
the time that it's returned.

Change-Id: I1393933eb5f0bc3f86646bf4d0acfc64626efbe0
2023-09-26 14:04:33 +02:00
Ivan Fraixedes
822a13570e
private/apigen: Document Endpoint fields
Add document comments to tall the fields of the Endpoint struct.

Change-Id: Ibcd534f26474fc73fb378a9f30ce7d9401217557
2023-09-26 14:04:32 +02:00
Ivan Fraixedes
9a06de9058 private/apigen: Add test for Types.All
Add a test for Types.All method because we may need to adjust the logic
in future commits and we want to detect what has changed for good or
bad.

Change-Id: I1db4bf67db3c87513cb9aeb7b942c6c132fc4dd1
2023-09-25 21:15:06 +00:00
Ivan Fraixedes
926076bffd private/apigen/example: Add description to endpoint
Add the description field to the endpoint to show where it ends up when
generating code.

Change-Id: I415f088dbf795656ed1ef042ed41ebf39a517692
2023-09-19 18:31:20 +00:00
Moby von Briesen
b4c95a3b28 private/apigen/tsgen: Set query params better
The old way did not properly handle escaping, e.g. if the value of a
query param contained `&` or `=` inside it. By using
url.searchParams.set, we can safely add these types of arguments to the
path.

Change-Id: I62d3883b14f9d5a517e4a3d58f019014b46fd1b4
2023-08-31 20:09:29 -04:00
Egon Elbre
273ebd61d7 private/apigen/example: make it nicer
Change-Id: I1bd779090a902ed2b99b3993dc7cf61fc250f10f
2023-08-28 15:13:46 +00:00
Ivan Fraixedes
2d8f396eeb private/apigen: Make API base path configurable
Previously the base path for the API was hardcoded to `/api` and the
specified version.

This was not obvious that the generated code was setting that base path
and it was not flexible for serving the API under a different path than
`/api`.

We will likely need to set a different base path if we pretend to serve
the new back office API that we are going to implement alongside the
current admin API until the new back office is fully implemented and
verified that works properly.

This commit also fix add the base path of the endpoints to the
documentation because it was even more confusing for somebody that wants
to use the API having to find out them through looking to the generated
code.

Change-Id: I6efab6b6f3d295129d6f42f7fbba8c2dc19725f4
2023-08-28 14:35:01 +00:00
Ivan Fraixedes
d34c1b6825 private/apigen: Generate docs & typescript for example
Generate the documentation and the typescript code for the example of
that we have to generate an API through the API generator.

The JSON Go struct field tags are needed because the typescript
generator require them, otherwise it panics.

The test had to be adjusted according to match the tags so Go consider
the structs the same type, otherwise, it doesn't compile.

Change-Id: I3e4ebff9174885c50ca2058b86b7ec60e025945c
2023-08-25 12:30:21 +00:00
Moby von Briesen
d76f059c55 private/apigen: Update generated doc links
Make the link more human-friendly - to contain the text of the group and
endpoint names.

Also link back to list of endpoints from each endpoint.

Change-Id: Ia3e2ebe20b58b5f60ecefe9d35fb8fd90dd4b4d7
2023-08-25 11:51:57 +00:00
Moby von Briesen
7fbabbcf16 private/apigen: Add list of endpoints with links
This change makes it easier for someone reading the documentation to see
a full list of supported endpoints, and have direct links to the
details.

Change-Id: I46e2f809cfa2760845898eaa3d99db9066d435ef
2023-08-24 02:45:46 +00:00
Ivan Fraixedes
b0d072270b private/apigen: Add comment about panics
Add a comment in 2 methods about they panic as it's specified for the
`MustWriteDocs`.

Change-Id: I15d5fa420af37318d828bc633bed62988f8b207b
2023-08-23 18:50:37 +00:00
Egon Elbre
b5e1e5a9e2 private/apigen: use typescript in docs markdown
Fixes https://github.com/storj/storj/issues/6187

Change-Id: I34016513f29a9538343f0909951ab1a4605bb585
2023-08-17 15:33:21 +00:00
Moby von Briesen
8acb1ee5bf private/apigen: Support basic doc generation
Add some code to generate a basic markdown file documenting a generated
API. Generate this document for the API in
satellite/console/consoleweb/consoleapi/gen.

The documentation is not completely correct, as it may include some
values in the request body that are not actually usable by the
requester. This can be fixed by making sure all types used within the
generated API are properly annotated with `json` tags.

Issue: https://github.com/storj/storj-private/issues/244

Change-Id: I57b259967fb0db8f548b6598a10c825da15ba723
2023-06-13 08:48:06 +00:00
Jeremy Wharton
4d823e8166 private/apigen: isolate TypeScript class generation
The code responsible for generating TypeScript classes has been
separated from the rest of the TypeScript generation code so that other
packages may take advantage of this functionality.

References #5494

Change-Id: I97eabd430bd6a5f748eafaf8b1d783977e75e660
2023-03-10 21:52:30 +00:00
Moby von Briesen
3d1007ae18 private/apigen: Change order of operations in api generation
Move the IsAuthenticated check until after initial parameter
parsing/validation. IsAuthenticated will be more expensive than
parsing/validation, so we should fail before auth if possible.

Change-Id: I96a020892eabcb750e8ec9ecc1d8b7d9bf8bf573
2023-01-11 16:54:53 +00:00
Egon Elbre
ff22fc7ddd all: fix deprecated ioutil commands
Change-Id: I59db35116ec7215a1b8e2ae7dbd319fa099adfac
2022-10-11 15:27:29 +00:00
Egon Elbre
794a4cc365 private/apigen: make code compile with 1.16
Our Test Versions still requires 1.16 to be compatible with our oldest
uplink versions. These changes make the code compile with 1.16.

Also, it makes go generate work in private/apigen/example.

Change-Id: Ib2f7493941a16f361328fe01d2be293f26123719
2022-09-02 16:34:46 +03:00
Egon Elbre
1343528a43 satellite/console/.../consoleapi/gen: ensure go generate works
Currently the paths were set relative to the root of the module,
however the code did not ensure that we are running relative to the
module directory.

Also, ensure typescript output corresponds to our styling.

Change-Id: I2b3cbd4ea8f2615e35c7b58c6fb8851669c47885
2022-09-02 16:34:46 +03:00
Jeremy Wharton
1613d37466 satellite/console: add endpoints for listing and revoking api keys
This change adds the following endpoints:
- projects/apikeys/{id}: returns a paged list of API keys for the
	project specified by the given ID
- apikeys/delete/{id}: deletes the API key specified by the given ID

Additionally, the API Go code generator has been given the ability to
process unsigned integer parameters.

Change-Id: I5ff24e012da24a3f06bea1ebb62bae6ff62f951a
2022-09-01 14:24:00 -05:00
Cameron
fa23e55654 {private,satellite}: add Typescript generation to REST API generation
github issue: https://github.com/storj/storj/issues/4984

Change-Id: Id3473a6642f7a4af80edda25a6242559efaf69e9
2022-08-18 16:37:48 +00:00
Jeremy Wharton
1f0638719e private/apigen,cmd/apigentest: add tests for generated API code
This change implements a unit test for ensuring proper
processing of requests and responses by generated API code.
Additionally, this change requires API handlers to explicitly receive
Monkit scopes rather than assuming that `mon` will always exist in the
generated API code's namespace.

Change-Id: Iea56f139f9dad0050b7d09ea765189280c3466f2
2022-08-15 16:48:41 +00:00
Jeremy Wharton
731fecd96f private/apigen: rework request parameter handling
- Previously unused struct Endpoint.Request now defines the form
	of the request body.
- Path parameters (e.g. "id" in "/delete/{id}") are defined in
	the Endpoint.PathParams field.
- Endpoint.Params has been renamed to Endpoint.QueryParams to
	eliminate confusion.

Change-Id: Ifef51ca2f362c33086f0e43e936d50b0fdd18aa1
2022-08-08 15:41:24 +00:00
Jeremy Wharton
5ce7d980af private/apigen: Make API generation deterministic
This change fixes the issue where the API generator would produce
different Go code for the same API definition upon each invocation
due to the random nature of map iteration.

Change-Id: I6770a10faf06311c24f541611c25d0b2b0f8e521
2022-06-17 12:06:08 -05:00