REFERENCE
Frequently asked questions
Find answers about model support, privacy, and validation. Your coding agent can read these answers through the API.
Preparing a request
What exact information must I provide?
The operations, the type and shape of each input, output, and weight, and what to optimize; weight values, input values, and measurements are optional, and each one permits more optimization.
A request has required facts and optional facts. A required fact defines the computation. An optional fact tells RunLocal more about the calls that are important to you. Each optional fact permits more optimization.
Model
| Fact | Required | What it gives |
|---|---|---|
| Each operation, and how the values connect | Yes | It defines the computation. Each node names one operator that has a defined meaning. |
| Each fixed setting and each constant | Yes | A node argument, an attribute, and a constant tensor are public. Without them an operation has no definition. |
| The type and shape of each input, output, and weight | Yes | A type is exact: an element type, and a shape of numbers or dimension names. |
| What to optimize | Yes | A target selects a function, one call, or a set of nodes or steps. |
| Weight values | No | They are private by default and stay on your machine. |
| Bounds on weight values | No | A replacement can use a bound that you state, such as finite values or a maximum. |
| The stored quantized form of a weight | No | The request states the codes, the scale, and the zero point. RunLocal can then keep the weight in that form. |
| Public or synthetic weight values | No | With values, RunLocal can assess quantization and compression. Synthetic values do not show accuracy. |
Inputs
| Fact | Required | What it gives |
|---|---|---|
| The range of each dimension that changes between calls | No | A dimension with no stated range has no limit. A narrow range permits more shape specialization. |
| Examples: the calls that are important to you | No | An example binds a dimension to one number or to a narrower range. RunLocal optimizes for these calls. |
| Measured counts | No | A count records an extent that depends on the input data, such as the number of active sparse pairs. |
| Input values | No | A sample file is public. A local name stays on your machine. Values show how much work depends on the data. |
A sample does not limit the calls that are legal. To make a promise about each call, write a dimension range or a constraint.
Device
A request does not name a device. You select the target device separately, so one request serves each device.
The request has no field for a distribution of values yet. State a bound, or supply a sample.
What stays on my machine?
Weight values and input values stay on your machine unless you declare them public; the request holds names, types, and shapes.
- A weight port is private by default. A private weight cannot be a sample file. An example binds it to a local name, and RunLocal does not follow that name.
- An input has the same choice. A sample file is public and is sent. A local name stays on your machine.
- A synthetic weight is made on your machine from a seed. No value is sent.
- A constant in a graph is public. Make the graph before a transformation that puts weight values into it. If you remove the source file later, the values stay in the graph. RunLocal cannot show that a graph has no private value, so review each constant before you send it.
- These are sent: the request, each graph file, each inline source that you include, and each source file that you attach. Each file has a stated size and digest.
- RunLocal does not parse, import, or run inline source, an adapter, or an implementation during a check or an upload. It does not fetch a link.
A checkpoint header is sufficient to write the weight ports. It gives the name, the type, and the shape of each tensor, and no value.
Must I convert my model to ONNX?
No: a graph uses ONNX operators, or Torch operators with a pinned environment.
No. A function body is a JSON graph, and the graph uses one operator system.
| System | When to use it |
|---|---|
| ONNX | You have an ONNX export. Each node uses a standard ONNX operator, a local function, or an operator extension. The ONNX file can stay attached as the source. |
| Torch | You have PyTorch code. Each node uses a public Torch callable, a public module, or inline source. The environment pins the exact Python version and each package version. |
The JSON graph defines the computation. An attached source file is a record of origin only.
RunLocal checks each Torch node against the catalog of the pinned release. A release with no catalog gives the result not_assessed, which is not a failure.
A large model is a set of graph functions and one steps body that calls them. A step that calls the same layer again can lift the weight names of that layer with one entry, so a request stays small.
Graphs and shapes
Are symbolic shapes allowed?
Yes: a dimension is a number, a name with a range, or ?, and you decide how much RunLocal can specialize for shapes, for input data, and for weights.
Yes. A dimension in a tensor type is a number, a name such as N, or ?. A name is the same extent in each place where it occurs in one function. A function states a range for a name, and it can state that one dimension is a sum of others.
You decide how much RunLocal can specialize. There are three kinds of specialization.
| Kind | What changes between calls | How you state it |
|---|---|---|
| Shape | The extents of the tensors | Write a number for a fixed extent. Write a name and a range for an extent that changes. In an example, bind the name to one number or to a narrower range. A narrow range permits more specialization. |
| Workload | The work depends on the input values, not only on the shapes. Examples are the active pairs of a sparse convolution rulebook, the occupied cells after grid pooling, and the tokens that each expert gets. | Give the extent its own dimension name and range. Record the measured count on your samples. Supply the input as a sample file or as a local value. |
| Weight | The values of the weights | A weight is private by default, and a replacement must be correct for each permitted value. This version does not put weight values into the computation. A value bound, a quantization overlay, and public or synthetic values each give RunLocal more to work with. |
A count is a measurement of your samples. It is not a promise about other calls. To make a promise, write a range or a constraint.
A relation between dimensions can be a sum. It cannot be a product yet. State a product, such as padded rows = 1024 × patches, in the note of the function.
Operators
What if I have a custom operator that I want to optimize?
RunLocal can optimize it when the request gives it one exact meaning: inline Torch source, an ONNX local function, or an operator extension with a specification.
Yes, when the request gives the operator one exact meaning. For each permitted input, the definition must give the outputs and the numerical differences that are permitted. A name, a link, or a standard operator with a similar name does not define an operator.
| Your operator | How to define it |
|---|---|
| Python code that calls Torch | Inline source in a Torch operator. The source and the pinned environment define it. For a call to another package, such as a sparse convolution library, pin that package in the environment. |
| A Torch module with parameters | A module definition with inline source. Each parameter binds to a weight port. |
| A composition of ONNX operators | An ONNX local function in the graph document. |
| A kernel with no Torch form and no ONNX form | An operator extension. Its specification states the inputs, the outputs, the attributes, the type rules, the shape rules, the boundary behavior, the numerical behavior, the permitted nondeterminism, and the state effects. A pinned reference implementation can be the authority in place of the specification. |
| Code that must not change | A black box body. RunLocal keeps each call, its arguments, and its order. It does not optimize the code. |
| A meaning that you do not know yet | An explicit unknown. RunLocal stores it and shows it. Each scope that uses it is not ready. RunLocal does not read it as an identity or as a pure operation. |
RunLocal does not run your source during a check. It reads the source as data.
Results
What does a passed check mean?
RunLocal found no contradiction in the facts that it can check; it does not show that the model is correct or that an optimized version is equivalent.
It means that RunLocal found no contradiction in the facts that it can check. It does not mean that the model is correct, or that an optimized version is equivalent.
| RunLocal checks | RunLocal does not establish |
|---|---|
| Strict JSON and the closed schema | That a graph computes the output shape that it declares |
| Each reference, file size, and digest | Numerical behavior |
| Graph structure and each port binding | Equivalence with your source code |
| Each operator definition against its catalog | Model accuracy |
| The types on each connection between functions | Speed on a device |
| The request digests |
A result has one of four states, and RunLocal does not join them into one pass or fail value. An invalid fact is a contradiction. An incomplete fact is missing. An unsupported fact uses an extension that RunLocal cannot read. A fact that is not assessed has no catalog or no device profile.
Put your own tests in the knowledge map as tested evidence. Give each entry an explicit outcome. A finite set of tests is evidence. It is not a proof for every input.
How do I put a result into my codebase?
A result obeys the contract in your request, so the code that calls the model does not change; your coding agent reads the .runlocal/ folder to connect the result to your code.
- Your request is the contract. It says what RunLocal can change and what must stay the same, for example the entrypoints, the inputs and outputs, and the exact weight shapes.
- Each result obeys that contract. You can use it instead of the original model, and the code that calls the model does not change.
- Keep the
.runlocal/folder. It records how the request connects to your code, for example the local names of your weights and inputs. Some of its files are private and stay on your machine. Your coding agent reads this folder to fit the result into your codebase.