Protocol Documentation

Table of Contents

• docarray.proto

  • DocListProto

  • DocProto

• jina.proto

  • DataRequestListProto

  • DataRequestProto

  • DataRequestProto.DataContentProto

  • DataRequestProtoWoData

  • EndpointsProto

  • HeaderProto

  • JinaInfoProto

  • JinaInfoProto.EnvsEntry

  • JinaInfoProto.JinaEntry

  • RelatedEntity

  • RestoreId

  • RestoreSnapshotCommand

  • RestoreSnapshotStatusProto

  • RouteProto

  • SingleDocumentRequestProto

  • SnapshotId

  • SnapshotStatusProto

  • StatusProto

  • StatusProto.ExceptionProto

  • RestoreSnapshotStatusProto.Status

  • SnapshotStatusProto.Status

  • StatusProto.StatusCode

  • JinaDataRequestRPC

  • JinaDiscoverEndpointsRPC

  • JinaExecutorRestore

  • JinaExecutorRestoreProgress

  • JinaExecutorSnapshot

  • JinaExecutorSnapshotProgress

  • JinaGatewayDryRunRPC

  • JinaInfoRPC

  • JinaRPC

  • JinaSingleDataRequestRPC

  • JinaSingleDocumentRequestRPC

• Scalar Value Types

Top

docarray.proto

DocListProto

this file is just a placeholder for the DA coming from jina._docarray dependency

DocProto

Top

jina.proto

DataRequestListProto

Represents a list of data requests This should be replaced by streaming

Field	Type	Label	Description
requests	DataRequestProto	repeated	requests in this list

DataRequestProto

Represents a DataRequest

Field	Type	Label	Description
header	HeaderProto		header contains meta info defined by the user
parameters	google.protobuf.Struct		extra kwargs that will be used in executor
routes	RouteProto	repeated	status info on every routes
data	DataRequestProto.DataContentProto		container for docs and groundtruths

DataRequestProto.DataContentProto

Field	Type	Label	Description
docs	docarray.DocListProto		the docs in this request
docs_bytes	bytes		the docs in this request as bytes

DataRequestProtoWoData

Field	Type	Label	Description
header	HeaderProto		header contains meta info defined by the user
parameters	google.protobuf.Struct		extra kwargs that will be used in executor
routes	RouteProto	repeated	status info on every routes

EndpointsProto

Represents the set of Endpoints exposed by an Executor

Field	Type	Label	Description
endpoints	string	repeated	list of endpoints exposed by an Executor
write_endpoints	string	repeated
schemas	google.protobuf.Struct		Dictionary containing input and output schema per endpoint

extra kwargs that will be used in executor |

HeaderProto

Represents a Header.

• The header’s content will be defined by the user request.

• It will be copied to the envelope.header

• In-flow operations will modify the envelope.header

• While returning, copy envelope.header back to request.header

Field	Type	Label	Description
request_id	string		the unique ID of this request. Multiple requests with the same ID will be gathered
status	StatusProto		status info
exec_endpoint	string	optional	the endpoint specified by @requests(on='/abc')
target_executor	string	optional	if set, the request is targeted to certain executor, regex strings
timeout	uint32	optional	epoch time in seconds after which the request should be dropped

JinaInfoProto

Field	Type	Label	Description
jina	JinaInfoProto.JinaEntry	repeated	information about the system running and package version information including jina
envs	JinaInfoProto.EnvsEntry	repeated	the environment variable setting

JinaInfoProto.EnvsEntry

Field	Type	Label	Description
key	string
value	string

JinaInfoProto.JinaEntry

Field	Type	Label	Description
key	string
value	string

RelatedEntity

Represents an entity (like an ExecutorRuntime)

Field	Type	Label	Description
id	string		unique id of the entity, like the name of a pod
address	string		address of the entity, could be an IP address, domain name etc, does not include port
port	uint32		port this entity is listening on
shard_id	uint32	optional	the id of the shard it belongs to, if it is a shard

RestoreId

An ID representation for the Restore Job

Field	Type	Label	Description
value	string

RestoreSnapshotCommand

Commands to restore an Executor from a snapshot file

Field	Type	Label	Description
snapshot_file	string		snapshot file

RestoreSnapshotStatusProto

Represents the status of a Restore.

Field	Type	Label	Description
id	RestoreId		An ID for the Restore Job.
status	RestoreSnapshotStatusProto.Status		status code

RouteProto

Represents a the route paths of this message as perceived by the Gateway start_time is set when the Gateway sends a message to a Pod end_time is set when the Gateway receives a message from a Pod thus end_time - start_time includes Executor computation, runtime overhead, serialization and network

Field	Type	Label	Description
executor	string		the name of the BasePod
start_time	google.protobuf.Timestamp		time when the Gateway starts sending to the Pod
end_time	google.protobuf.Timestamp		time when the Gateway received it from the Pod
status	StatusProto		the status of the execution

SingleDocumentRequestProto

Represents a Single Document DataRequest

Field	Type	Label	Description
header	HeaderProto		header contains meta info defined by the user
parameters	google.protobuf.Struct		extra kwargs that will be used in executor
routes	RouteProto	repeated	status info on every routes
document	docarray.DocProto		the document in this request

SnapshotId

An ID representation for the Snapshot Job

Field	Type	Label	Description
value	string

SnapshotStatusProto

Represents the status of a Snapshot.

Field	Type	Label	Description
id	SnapshotId		An ID for the Snapshot Job.
status	SnapshotStatusProto.Status		status code
snapshot_file	string		snapshot file

StatusProto

Represents a Status

Field	Type	Label	Description
code	StatusProto.StatusCode		status code
description	string		error description of the very first exception
exception	StatusProto.ExceptionProto		the details of the error

StatusProto.ExceptionProto

Field	Type	Label	Description
name	string		the class name of the exception
args	string	repeated	the list of arguments given to the exception constructor.
stacks	string	repeated	the exception traceback stacks
executor	string		the name of the executor bind to that Executor (if applicable)

RestoreSnapshotStatusProto.Status

Name	Number	Description
STATE_UNSPECIFIED	0
RUNNING	1	Resource allocation has been successful. Restore Job is RUNNING.
SUCCEEDED	2	Resource Job has finished successfully.
FAILED	3	Resource Job has failed.
NOT_FOUND	6	If the specific restore id is unknown or not found

SnapshotStatusProto.Status

Name	Number	Description
STATE_UNSPECIFIED	0
QUEUED	1	Snapshot Job is admitted (validated and persisted) and waiting for resources.
SCHEDULED	2	Snapshot Job is scheduled to run as soon as resource allocation is ready.
RUNNING	3	Resource allocation has been successful. Snapshot Job is RUNNING.
SUCCEEDED	4	Snapshot Job has finished successfully.
FAILED	5	Snapshot Job has failed.
NOT_FOUND	6	If the specific snapshot id is unknown or not found

StatusProto.StatusCode

Name	Number	Description
SUCCESS	0	success
ERROR	1	error

JinaDataRequestRPC

jina gRPC service for DataRequests.

Method Name	Request Type	Response Type	Description
process_data	DataRequestListProto	DataRequestProto	Used for passing DataRequests to the Executors

JinaDiscoverEndpointsRPC

jina gRPC service to expose Endpoints from Executors.

Method Name	Request Type	Response Type	Description
endpoint_discovery	.google.protobuf.Empty	EndpointsProto

JinaExecutorRestore

jina gRPC service to trigger a restore at the Executor Runtime.

Method Name	Request Type	Response Type	Description
restore	RestoreSnapshotCommand	RestoreSnapshotStatusProto

JinaExecutorRestoreProgress

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name	Request Type	Response Type	Description
restore_status	RestoreId	RestoreSnapshotStatusProto

JinaExecutorSnapshot

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name	Request Type	Response Type	Description
snapshot	.google.protobuf.Empty	SnapshotStatusProto

JinaExecutorSnapshotProgress

jina gRPC service to trigger a snapshot at the Executor Runtime.

Method Name	Request Type	Response Type	Description
snapshot_status	SnapshotId	SnapshotStatusProto

JinaGatewayDryRunRPC

jina gRPC service to expose Endpoints from Executors.

Method Name	Request Type	Response Type	Description
dry_run	.google.protobuf.Empty	StatusProto

JinaInfoRPC

jina gRPC service to expose information about running jina version and environment.

Method Name	Request Type	Response Type	Description
_status	.google.protobuf.Empty	JinaInfoProto

JinaRPC

jina streaming gRPC service.

Method Name	Request Type	Response Type	Description
Call	DataRequestProto stream	DataRequestProto stream	Pass in a Request and a filled Request with matches will be returned.

JinaSingleDataRequestRPC

jina gRPC service for DataRequests. This is used to send requests to Executors when a list of requests is not needed

Method Name	Request Type	Response Type	Description
process_single_data	DataRequestProto	DataRequestProto	Used for passing DataRequests to the Executors

JinaSingleDocumentRequestRPC

jina gRPC service for DataRequests. This is used to send requests to Executors when a list of requests is not needed

Method Name	Request Type	Response Type	Description
stream_doc	SingleDocumentRequestProto	SingleDocumentRequestProto stream	Used for streaming one document to the Executors

Scalar Value Types

.proto Type	Notes	C++	Java	Python	Go	C#	PHP	Ruby
double		double	double	float	float64	double	float	Float
float		float	float	float	float32	float	float	Float
int32	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint32 instead.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
int64	Uses variable-length encoding. Inefficient for encoding negative numbers – if your field is likely to have negative values, use sint64 instead.	int64	long	int/long	int64	long	integer/string	Bignum
uint32	Uses variable-length encoding.	uint32	int	int/long	uint32	uint	integer	Bignum or Fixnum (as required)
uint64	Uses variable-length encoding.	uint64	long	int/long	uint64	ulong	integer/string	Bignum or Fixnum (as required)
sint32	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int32s.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sint64	Uses variable-length encoding. Signed int value. These more efficiently encode negative numbers than regular int64s.	int64	long	int/long	int64	long	integer/string	Bignum
fixed32	Always four bytes. More efficient than uint32 if values are often greater than 2^28.	uint32	int	int	uint32	uint	integer	Bignum or Fixnum (as required)
fixed64	Always eight bytes. More efficient than uint64 if values are often greater than 2^56.	uint64	long	int/long	uint64	ulong	integer/string	Bignum
sfixed32	Always four bytes.	int32	int	int	int32	int	integer	Bignum or Fixnum (as required)
sfixed64	Always eight bytes.	int64	long	int/long	int64	long	integer/string	Bignum
bool		bool	boolean	boolean	bool	bool	boolean	TrueClass/FalseClass
string	A string must always contain UTF-8 encoded or 7-bit ASCII text.	string	String	str/unicode	string	string	string	String (UTF-8)
bytes	May contain any arbitrary sequence of bytes.	string	ByteString	str	[]byte	ByteString	string	String (ASCII-8BIT)