text:
"Number must be greater than 0" Errors

Hello! I'm likely doing something wrong here, but, trying to work through a sample application that is processing a single small document and then querying the index for information. I'm attempting to do this all locally using Ollama - but I have not been able to to get past this set of errors.
Any ideas or suggestions on what to try?
Number must be greater than 0 (["chunkSize"])
Number must be greater than or equal to 0 (["chunkOverlap"])
Chunk overlap must be less than chunk size. ([])

Running this in Node v22.9.0.
Example code:
import {
    Settings,
    Ollama,
    VectorStoreIndex,
    SimpleDirectoryReader,
    OllamaEmbedding,
} from 'llamaindex'

const MODEL_NAME = 'llama3.2'
const ollama = new Ollama({
    model: MODEL_NAME,
    config: {},
    options: { temperature: 0.5 },
})
const embedModel = new OllamaEmbedding({
    model: 'mxbai-embed-large',
    options: { temperature: 0.5 },
})

Settings.llm = ollama
Settings.embedModel = embedModel

async function main() {
    const dataDocs = await new SimpleDirectoryReader().loadData({
        directoryPath: './data',
    })

    console.log(`Loaded ${dataDocs.length} doument(s).`)

    const index = await VectorStoreIndex.fromDocuments(dataDocs, {
        logProgress: true,
    })
    const retriever = index.asRetriever()
    const queryEngine = index.asQueryEngine({
        retriever: retriever,
    })
    const stream = await queryEngine.query({
        query: '{{Question}}',
        stream: true,
    })

    for await (const chunk of stream) {
        process.stdout.write(chunk.delta)
    }
}
main().catch((e) => console.error(e))

Using node parser on documents...
Finished parsing documents.
getting embedding progress: 0 / 1
ZodError: [
  {
    "code": "too_small",
    "minimum": 0,
    "type": "number",
    "inclusive": false,
    "exact": false,
    "message": "Number must be greater than 0",
    "path": [
      "chunkSize"
    ]
  },
  {
    "code": "too_small",
    "minimum": 0,
    "type": "number",
    "inclusive": true,
    "exact": false,
    "message": "Number must be greater than or equal to 0",
    "path": [
      "chunkOverlap"
    ]
  },
  {
    "code": "custom",
    "message": "Chunk overlap must be less than chunk size.",
    "path": []
  }
]
    at get error (file:///./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:587:31)
    at ZodEffects.parse (file:///./node_modules/.pnpm/zod@3.23.8/node_modules/zod/lib/index.mjs:692:22)
    at new SentenceSplitter (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/node-parser/dist/index.js:318:57)
    at PromptHelper.getTextSplitterGivenPrompt (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/indices/dist/index.js:72:16)
    at PromptHelper.repack (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/indices/dist/index.js:82:35)
    at CompactAndRefine.getResponse (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/response-synthesizers/dist/index.js:209:44)
    at CompactAndRefine.synthesize (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/response-synthesizers/dist/index.js:31:35)
    at RetrieverQueryEngine._query (file:///./node_modules/.pnpm/llamaindex@0.6.13_@aws-sdk+client-sso-oidc@3.662.0_@aws-sdk+client-sts@3.662.0__@aws-sdk+cred_ottdx7krjsm3z4s7c6pq2277m4/node_modules/llamaindex/dist/engines/query/RetrieverQueryEngine.js:15:49)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async RetrieverQueryEngine.query (file:///./node_modules/.pnpm/@llamaindex+core@0.2.8_@aws-crypto+sha256-js@5.2.0_@xenova+transformers@2.17.2_js-tiktoken@1._bmi77gdnidaxf3lqzmibpj7squ/node_modules/@llamaindex/core/query-engine/dist/index.js:405:26)
    at async main (C:\Depot\AI\mtg-processor\index.ts:69:20) {
  issues: [
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: false,
      exact: false,
      message: 'Number must be greater than 0',
      path: [Array]
    },
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: true,
      exact: false,
      message: 'Number must be greater than or equal to 0',
      path: [Array]
    },
    {
      code: 'custom',
      message: 'Chunk overlap must be less than chunk size.',
      path: []
    }
  ],
  addIssue: [Function (anonymous)],
  addIssues: [Function (anonymous)],
  errors: [
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: false,
      exact: false,
      message: 'Number must be greater than 0',
      path: [Array]
    },
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: true,
      exact: false,
      message: 'Number must be greater than or equal to 0',
      path: [Array]
    },
    {
      code: 'custom',
      message: 'Chunk overlap must be less than chunk size.',
      path: []
    }
  ]
}

This is werid, I can run it successfully. Make sure ./data directory load document successfully

Oh, i see the error when there's only one document

I can see the problem here is context size is even smaller than prompt length size

Please set num_ctx to the correct value, because we cannot help you to get the correct llm metadata
const ollama = new Ollama({
	model: MODEL_NAME,
	config: {},
	options: {
		temperature: 0.5,
	  num_ctx: 128_000,
	},
})

I seem to get the same error with more than one document and longer documents. It seems like the chunk size and chunk overlap are being seen as zero (or not set), but I'm not sure why.

Please set num_ctx to the correct value, because we cannot help you to get the correct llm metadata

I set this, and it gives the same results.

I set this, and it gives the same results.

So the reason is from loader side, you might load a too large content into one Document.
You can run text spliiter after load the document
const dataDocs = await new SimpleDirectoryReader().loadData({
  directoryPath: './empty',
})
const textSplitter = new SentenceSplitter()
const docs = await textSplitter(dataDocs)

So the reason is from loader side, you might load a too large content into one Document.

This document is short (4kb). The loader code seems to run fine but fails after it requests embeddings (though embeddings return manually if i pass the content to Ollama manually via API.

Updated to llamaindex v0.6.15 and still encounter the same errors.

I set this up to store the index to disk and the system is generating what appears to be valid vector stores, etc. doc_store.json, index_store.json, and vector_store.json all get populated. But running an actual query still fails with the same errors.

From what I've been able to figure out so far is it appears there is a SentenceSplitter getting setup somewhere that is missing or has bad configuration... But I'm not specifically sure where that is happening inside of the query chain. But calling query on an index fails before making a call to the LLM - so it seems like an internal Llamaindex configuration issue.

I'm also having the same issue with Ollama.  My goal is to use Ollama and local embeddings instead of OpenAI
Code:
import fs from "node:fs/promises";

import {
  Document,
  MetadataMode,
  NodeWithScore,
  VectorStoreIndex,
  Ollama,
  OllamaEmbedding,
  Settings,
  HuggingFaceEmbedding,
  OpenAI
} from "llamaindex";

const MODEL_NAME = 'llama3';

const ollama = new Ollama({ model: "llama3" });
const embedModel = new HuggingFaceEmbedding({
  modelType: "BAAI/bge-small-en-v1.5",
  quantized: false,
});

Settings.llm = ollama;
Settings.embedModel = embedModel;

async function main() {
  // Load essay from abramov.txt in Node
  const path = "node_modules/llamaindex/examples/abramov.txt";

  const essay = await fs.readFile(path, "utf-8");

  // Create Document object with essay
  const document = new Document({ text: essay, id_: path });

  // Split text and create embeddings. Store them in a VectorStoreIndex
  const index = await VectorStoreIndex.fromDocuments([document]);

  // Query the index
  const queryEngine = index.asQueryEngine();
  const { response, sourceNodes } = await queryEngine.query({
    query: "What did the author do in college?",
  });

  // Output response with sources
  console.log(response);

  if (sourceNodes) {
    sourceNodes.forEach((source: NodeWithScore, index: number) => {
      console.log(
        `\n${index}: Score: ${source.score} - ${source.node.getContent(MetadataMode.NONE).substring(0, 50)}...\n`,
      );
    });
  }
}

main().catch(console.error);

I get this error
(node:38005) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
ZodError: [
  {
    "code": "too_small",
    "minimum": 0,
    "type": "number",
    "inclusive": false,
    "exact": false,
    "message": "Number must be greater than 0",
    "path": [
      "chunkSize"
    ]
  },
  {
    "code": "too_small",
    "minimum": 0,
    "type": "number",
    "inclusive": true,
    "exact": false,
    "message": "Number must be greater than or equal to 0",
    "path": [
      "chunkOverlap"
    ]
  },
  {
    "code": "custom",
    "message": "Chunk overlap must be less than chunk size.",
    "path": []
  }
]
    at get error (LlamaIndexTS/node_modules/zod/lib/types.js:55:31)
    at ZodEffects.parse (LlamaIndexTS/node_modules/zod/lib/types.js:160:22)
    at new SentenceSplitter (LlamaIndexTS/node_modules/@llamaindex/core/node-parser/dist/index.cjs:320:67)
    at PromptHelper.getTextSplitterGivenPrompt (LlamaIndexTS/node_modules/@llamaindex/core/indices/dist/index.cjs:74:16)
    at PromptHelper.repack (LlamaIndexTS/node_modules/@llamaindex/core/indices/dist/index.cjs:84:35)
    at CompactAndRefine.getResponse (LlamaIndexTS/node_modules/@llamaindex/core/response-synthesizers/dist/index.cjs:211:44)
    at CompactAndRefine.synthesize (LlamaIndexTS/node_modules/@llamaindex/core/response-synthesizers/dist/index.cjs:33:35)
    at RetrieverQueryEngine._query (LlamaIndexTS/node_modules/llamaindex/dist/cjs/engines/query/RetrieverQueryEngine.js:30:45)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async RetrieverQueryEngine.query (LlamaIndexTS/node_modules/@llamaindex/core/query-engine/dist/index.cjs:407:26)
    at main (LlamaIndexTS/main.ts:54:37) {
  issues: [
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: false,
      exact: false,
      message: 'Number must be greater than 0',
      path: [Array]
    },
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: true,
      exact: false,
      message: 'Number must be greater than or equal to 0',
      path: [Array]
    },
    {
      code: 'custom',
      message: 'Chunk overlap must be less than chunk size.',
      path: []
    }
  ],
  addIssue: [Function (anonymous)],
  addIssues: [Function (anonymous)],
  errors: [
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: false,
      exact: false,
      message: 'Number must be greater than 0',
      path: [Array]
    },
    {
      code: 'too_small',
      minimum: 0,
      type: 'number',
      inclusive: true,
      exact: false,
      message: 'Number must be greater than or equal to 0',
      path: [Array]
    },
    {
      code: 'custom',
      message: 'Chunk overlap must be less than chunk size.',
      path: []
    }
  ]
}

Everything works fine when I use OpenAI as the model instead an Ollama model
const openai = new OpenAI({ model: "gpt-3.5-turbo", temperature: 0 });

const embedModel = new HuggingFaceEmbedding({
  modelType: "BAAI/bge-small-en-v1.5",
  quantized: false,
});

Settings.llm = openai
Settings.embedModel = embedModel

working on it

repo: run-llama/LlamaIndexTS


text:
[Bug]: Different Markdown modules don't deal with indented fenced code blocks

Bug Description
I was doing some tests with the various:

MarkdownElementNodeParser
MarkdownNodeParser
CodeSplitter
MarkdownReader

To see how I could get the better from them... when I detected that, for some Markdown files, I was missing a lot of nodes after some given points.
Ultimately, and apart from the different implementations in all the modules above (they really should be sharing code, IMO) I traced the problem to lines like this in many of them:
if line.startswith("```"):

And the problem being that it's legit to write those code blocks with 4x indentation spaces (whenever they are within list, for example).
So this issue is about to consider when those conditions in different parsers/readers could be changed to, simply:
if line.lstrip().startswith("```"):

Making all them immune to any indentation.
Version
0.10.23
Steps to Reproduce
Just try to parse any Markdown with various headers and some indented code blocks, results are crazy, the 3 problems commented in the example can be easily reproduced. With the proposed trim all them should be fixed.
Some text
# Header
## Header 2
### Header 3
```txt
Non indented block code
```
A list begins here:

* Element 1

    ```txt
    # has some indented code, but it's not handled as that.
    ```
* Element 2

```txt
    # also has some code, but unbalanced fences (different number of spaces). Everything after this is considered code block!
 ```

* Element 3
* Element 4
### Another header 3
 ```txt
# has some wrongly indented fence, and leads to incorrect header detection.
```

## Another header 2

Edited: I've added the markdown above with non-fenced code block, so it's easier to keep all those backticks.
Relevant Logs/Tracbacks
No response

As commented in the description, differences aside (between reader, parsers...) all these are potentially affected by the same issue:
$ ag 'startswith.*```'
llama-index-legacy/llama_index/legacy/node_parser/relational/markdown_element.py
78:            if line.startswith("```"):

llama-index-legacy/llama_index/legacy/node_parser/file/markdown.py
69:            if line.startswith("```"):

llama-index-legacy/llama_index/legacy/output_parsers/utils.py
97:        if candidate.startswith("```"):

llama-index-core/llama_index/core/node_parser/relational/markdown_element.py
52:            if line.startswith("```"):

llama-index-core/llama_index/core/node_parser/relational/llama_parse_json_element.py
111:                if line.startswith("```"):

llama-index-core/llama_index/core/node_parser/file/markdown.py
68:            if line.startswith("```"):

llama-index-core/llama_index/core/output_parsers/utils.py
97:        if candidate.startswith("```"):

I welcome a PR to change this (minus llama-index-legacy, that package shouldn't be changed

I welcome a PR to change this (minus llama-index-legacy, that package shouldn't be changed

I've created a draft: https://github.com/run-llama/llama_index/pull/12393
It only modifies MarkdownNodeParser. If looking good, I can follow the same approach with MarkdownElementNodeParser (not sure if there are others to look).
Ciao :-)

I've commented @ https://github.com/run-llama/llama_index/pull/12393. Impossible (to me!) to fix MarkdownElementNodeParseror MarkdownReader. So I've just focussed on MarkdownNodeParser.
Ciao :-)

source: gharchive/issue

license: MIT

license_type: permissive

license_source: github-api


text:
feat(annotations): add global annotations

what

Allow adding global annotations to created resources
Linted some files to make reading easier

why
Requested feature.
tests
Templates were generated against the main branch and this branch and compared to ensure there were no extra changes
→ diff --color ci-init-config-values-main.yaml ci-init-config-values-new.yaml

113a114
>
127,129c128,130
<         checksum/config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
<         checksum/repo-config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
<         checksum/init-config: da4123f44ff227abfe55354cd7872ec561edf6bc58885cca89bea77d6f533a21
---
>         checksum/config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
>         checksum/repo-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
>         checksum/init-config: e035a1806c7275d6e396d5c7d9b2b0bd51252a161e68ef3e652ee0fc5e6576d9

→ diff --color ci-values-main.yaml ci-values-new.yaml

77a78
>
91,92c92,93
<         checksum/config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
<         checksum/repo-config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
---
>         checksum/config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
>         checksum/repo-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b

Then I tested specifying an annotation and a secret:
→ helm template atlantis ./charts/atlantis/ --set api.secret=aaa --set extraAnnotations.aaa=bbb | diff --color old.yaml

-12a13,30
>   annotations:
>     aaa: bbb
> ---
> # Source: atlantis/templates/secret-api.yaml
> apiVersion: v1
> kind: Secret
> metadata:
>   name: atlantis-api
>   labels:
>     app: atlantis
>     chart: atlantis-4.21.1
>     helm.sh/chart: atlantis-4.21.1
>     release: atlantis
>     heritage: Helm
>   annotations:
>     aaa: bbb
> data:
>   apisecret: YWFh
24a43,44
>   annotations:
>     aaa: bbb
53a74,75
>   annotations:
>     aaa: bbb
75a98,100
>
>   annotations:
>     aaa: bbb
89,90c114,115
<         checksum/config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
<         checksum/repo-config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
---
>         checksum/config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
>         checksum/repo-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
118a144,148
>           - name: ATLANTIS_API_SECRET
>             valueFrom:
>               secretKeyRef:
>                 name: atlantis-api
>                 key: apisecret
165a196,197
>   annotations:
>     aaa: bbb

references
Closes https://github.com/runatlantis/helm-charts/issues/140

@roytev I love these unit tests 😍
FAIL  test statefulset	charts/atlantis/tests/statefulset_test.yaml
	- default values

		- asserts[12] `equal` fail
			Template:	atlantis/templates/statefulset.yaml
			DocumentIndex:	0
			Path:	spec.template.metadata.annotations
			Expected to equal:
				checksum/config: e3b0c44298fc1c149afbf4c8996fb92427ae41e[46](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:47)49b934ca495991b7852b855
				checksum/repo-config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
			Actual:
				checksum/config: 01ba[47](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:48)19c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
				checksum/repo-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c058ef8f9805daca546b
			Diff:
				--- Expected
				+++ Actual
				@@ -1,3 +1,3 @@
				-checksum/config: e3b0c44298fc1c1[49](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:50)afbf4c8996fb92427ae41e4649b934ca495991b7852b855
				-checksum/repo-config: e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855
				+checksum/config: 01ba4719c80b6fe911b091a7c0[51](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:52)24b64eeece964e09c058ef8f9805daca[54](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:55)6b
				+checksum/repo-config: 01ba4719c80b6fe911b091a7c05124b64eeece964e09c0[58](https://github.com/runatlantis/helm-charts/actions/runs/8072357303/job/22053924032?pr=356#step:8:59)ef8f9805daca546b

source: gharchive/pull-request

repo: runatlantis/helm-charts


text:
[Update] - Varlamore House Thieving - v1.4.0 - Sololegends

Release v1.4.0
Added time since bonus chest and other minor changes

Added overlay option for time since last bonus chest
Added option to turn off the flashing of the distracted citizen icon
Added option to turn off distracted icon whilst in a house
Bettered the owner left overlay rest detection
Resolved some edge cases for distraction notifications re-emitting when not needed

Implements Issue: https://github.com/sololegends/runelite-varlamore-house-thieving/issues/10
Resolves Issue: https://github.com/sololegends/runelite-varlamore-house-thieving/issues/9

Damn, what a rookie mistake. Thanks for pointing it out! Fixing it now.

Resolved!

Edit: can I not have the spot bugs in there? that is a development aspect not a runtime dependency.

Since it's a gradle plugin, it has more scope than the test classpath, so it must pass dependency verification too

authors: iProdigy sololegends

repo: runelite/plugin-hub


text:
Bank placeholder value popup inconsistency

Type
Incorrect behavior
Operating System
Windows
OS Version
Windows 10
Bug description
When you hover over a bank placeholder, no ge/ha value is shown. But when you grab another item or placeholder and hover it over a placeholder, it gives the value. Whether placeholder should show value or not, this is an inconsistency.
Screenshots or videos

RuneLite version
Runelite version: 32 bit, 1.10.14
Launcher version: 2.6.7
Osrs revision: 217
Logs
No response

Hey, wondering what other plugins you might have on, since when I try to replicate this I only get the weight pop up and not the item values. Granted, I am on Windows 11 but I don't think that should change much with how the program operates.

repo: runelite/runelite


text:
Option to add a sky/fog

One of the biggest problems I have with OSRS is the black void constantly in the background. It is quite ugly to look at and even scares me a little bit for some reason lol.
A good solution to this could be to have something similar to OSBuddy, where the black has been replaced by a sky. Or another idea could be to have a fog. Or both as options?
This is the "black void" I'm referring to.

This is how OSBuddy resolves the issue.

OSBuddy also increases the render distance which would also be nice for Runelite, but I believe the increased render distance is going to be implemented by Jagex eventually. However the black void is still going to be a problem, so I hope Runelite can add a Sky/Fog feature similar to OSBuddy.

That's OSBuddy OpenGL. An official OpenGL version is coming to PC some time after the mobile release, then more options will be available.

Duplicate #3941

Duplicate #3941

The duplicate you're referencing was ignored. Is this being worked on at all?

no

no

cool, back to osbuddy for the time being then

authors: Abextm IamAPancake ltvill raiyni


text:
fix: add ci.yaml to exclude from ci-docs triggers

Issue #, if available:
Description of changes:
Add ci.yaml to exclude from ci-docs triggers
Testing done:

[ ] I've reviewed the guidance in CONTRIBUTING.md

License Acceptance
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Is the PR title wrong? I see more changes than just excluding ci.yaml in ci-docs

repo: runfinch/finch


text:
BUG: 单层结构输出数据有误

问题
hi cai, 我在尝试使用的时候发现一个问题，单层的结构并没有正确的输出，运行 TestParse 用例。

排查问题（fix：建议）

增加了一个判断（暂时的） 正常工作

感谢 @Lucareful

authors: Lucareful runningzyp


text:
Could this repo be applied to train own data?

Could this repo be applied to train own data except for COCO, VOC? Is it OK that I have a dataset with binary classes? It is appreciated if there is more guidance.

I guess you have to build your dataloader for your dataset. I actually port it from other repository so I never changed the dataloading part,

Thanks for quick reply. I will try.

authors: LouieYang ruotianluo

repo: ruotianluo/pytorch-faster-rcnn


text:
Decide whether to wait for tokio reform

The first piece of the tokio reform was just released: https://tokio.rs/blog/2018-02-tokio-reform-shipped/
At this point, however, neither futures, nor mio or hyper have had the chance to release updates that align them with the new tokio crate.
The blog post hints at more changes to happen within the next 6-8 weeks.
We should observe how the situation develops, and decide whether we want to ship 0.32.0 for the "old" or the new tokio ecosystem.

I am still not up to speed on tokio, let alone the tokio reforms done and changes coming, but waiting six weeks for another release is a bit long for my tastes.
Would a README note as suggested above be enough notice about Rusoto's use of tokio and friends?

I suppose so, yeah.
As long as we can try to push out a release soon after the changes to future and hyper are released, that should be okay (I'm keen to drop the "custom" default reactor in rusoto for the new one in tokio).
Is this what we can record as the outcome here?

we will not wait for futures 0.2 (and associated hyper 0.12) before releasing Rusoto 0.32.0
we will call out clearly in the README that Rusoto 0.32.0 is not compatible with the new tokio crate

When can we cut a new release?

Here's the list of issues that are part of the 0.32 milestone: https://github.com/rusoto/rusoto/milestone/3

@srijs Those two items look right to me. 👍

authors: matthewkmayer srijs xrl

repo: rusoto/rusoto

license: mit

license_source: bigquery


text:
[Question] Are parameters supported for ALTER TABLE statements?

I've been trying to execute the below command for a couple hours now, using named parameters and positional parameters, encapsulating the parameters in parentheses as some of the documentation seems to do and not.
ALTER TABLE entries ADD COLUMN :name :type;

self.conn.execute("ALTER TABLE entries ADD COLUMN :name :type;", &[(":name", field.name.as_str()), (":type", type_full)]);

I get a syntax error at the :name (or opening parenthesis if using that). I've tried executing using the execute function and preparing a statement to execute separately. Given that similar code works with the INSERT command, and when hardcoding the name and type above, I can only guess that there's some issue in executing ALTER TABLE with parameters but there doesn't seem to be any note of this in the documentation. It's a statement, per the sqlite docs, and there doesn't seem to be a different specialized statement for table-altering commands in the rusqlite docs as far as I found.
Am I just blind and missing something obvious? Thanks in advance!

Nope, this is not supported. One way of thinking of this is that the value parameters evaluate are always expressions (in this case one evaluating to a string), and not identifiers. Whereas the column name must be an identifier -- perhaps you can think of it analgously to let "foo" = 3.
(That said, this explanation is somewhat handwavey and is not really accurate -- the true answer is probably just that it's a syntax error, and I'm making a guess to why that might be)

authors: Endominus thomcc

repo: rusqlite/rusqlite


text:
Bootstrap error EntryExists (FileSystemError): Unable to move/copy '...' because target '...' already exists at destination.

Just got this error today, and seems to happen every time I try to open up VSCode. I'm using VSCode Remote - SSH, connecting to a NixOS 21.05.3065.b3083bc6933 machine.
I haven't changed any configurations recently, so I'm not sure what could have caused this. Perhaps an extension version upgrade? Happy to provide any other diagnostics.
"Rust Analyzer Client" logs:
INFO [9/27/2021, 7:38:25 PM]: Extension version: 0.2.760
INFO [9/27/2021, 7:38:25 PM]: Using configuration {
  cargoRunner: null,
  runnableEnv: null,
  inlayHints: {
    enable: true,
    smallerHints: true,
    chainingHints: true,
    maxLength: 25,
    parameterHints: true,
    typeHints: true
  },
  updates: { channel: 'stable', askBeforeDownload: false },
  server: { path: null, extraEnv: null },
  trace: { server: 'off', extension: false },
  debug: {
    engine: 'auto',
    sourceFileMap: {
      '/rustc/<id>': '${env:USERPROFILE}/.rustup/toolchains/<toolchain-id>/lib/rustlib/src/rust'
    },
    openDebugPane: false,
    engineSettings: {}
  },
  assist: {
    importGranularity: 'crate',
    importEnforceGranularity: false,
    importPrefix: 'plain',
    importGroup: true,
    allowMergingIntoGlobImports: true
  },
  callInfo: { full: true },
  cargo: {
    autoreload: true,
    allFeatures: false,
    unsetTest: [ 'core' ],
    features: [],
    runBuildScripts: true,
    useRustcWrapperForBuildScripts: true,
    noDefaultFeatures: false,
    target: null,
    noSysroot: false
  },
  checkOnSave: {
    enable: true,
    allFeatures: null,
    allTargets: true,
    command: 'check',
    noDefaultFeatures: null,
    target: null,
    extraArgs: [],
    features: null,
    overrideCommand: null
  },
  completion: {
    addCallArgumentSnippets: true,
    addCallParenthesis: true,
    postfix: { enable: true },
    autoimport: { enable: true },
    autoself: { enable: true }
  },
  diagnostics: {
    enable: true,
    enableExperimental: true,
    disabled: [],
    remapPrefix: {},
    warningsAsHint: [],
    warningsAsInfo: []
  },
  experimental: { procAttrMacros: false },
  files: { watcher: 'client', excludeDirs: [] },
  highlightRelated: {
    references: true,
    exitPoints: true,
    breakPoints: true,
    yieldPoints: true
  },
  highlighting: { strings: true },
  hover: { documentation: true, linksInHover: true },
  hoverActions: {
    debug: true,
    enable: true,
    gotoTypeDef: true,
    implementations: true,
    references: false,
    run: true
  },
  joinLines: {
    joinElseIf: true,
    removeTrailingComma: true,
    unwrapTrivialBlock: true,
    joinAssignments: true
  },
  lens: {
    debug: true,
    enable: true,
    implementations: true,
    run: true,
    methodReferences: false,
    references: false,
    forceCustomCommands: true
  },
  linkedProjects: [],
  lruCapacity: null,
  notifications: { cargoTomlNotFound: true },
  procMacro: { enable: true, server: null },
  runnables: { overrideCargo: null, cargoExtraArgs: [] },
  rustcSource: null,
  rustfmt: {
    extraArgs: [],
    overrideCommand: null,
    enableRangeFormatting: false
  },
  workspace: { symbol: { search: { scope: 'workspace', kind: 'only_types' } } }
}
INFO [9/27/2021, 7:38:25 PM]: PersistentState: {
  lastCheck: undefined,
  nightlyReleaseId: undefined,
  serverVersion: undefined
}
INFO [9/27/2021, 7:38:28 PM]: Renamed old server binary /home/skainswo/.vscode-server/data/User/globalStorage/matklad.rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu to /home/skainswo/.vscode-server/data/User/globalStorage/matklad.rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu-stale-c30fce0c27
INFO [9/27/2021, 7:38:29 PM]: Removed old server binary /home/skainswo/.vscode-server/data/User/globalStorage/matklad.rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu-stale-c30fce0c27
ERROR [9/27/2021, 7:38:30 PM]: Bootstrap error EntryExists (FileSystemError): Unable to move/copy 'vscode-remote://ssh-remote+doodoo/home/skainswo/.vscode-server/data/User/globalStorage/matklad.rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu' because target 'vscode-remote://ssh-remote+doodoo/home/skainswo/.vscode-server/data/User/globalStorage/matklad.rust-analyzer/rust-analyzer-x86_64-unknown-linux-gnu-orig' already exists at destination.
    at _handleError (/home/skainswo/.vscode-server/bin/7f6ab5485bbc008386c4386d08766667e155244e/out/vs/server/remoteExtensionHostProcess.js:94:159993)
    at /home/skainswo/.vscode-server/extensions/matklad.rust-analyzer-0.2.760/out/main.js:108:47 {
  code: 'FileExists'
}

On NixOS, we run patchelf on a temporary copy of the server binary and we didn't overwrite a previous copy if it was left there. If you haven't removed that file yet, you might want to give #10422 a try. It should be available in the next nightly and in the Monday release.

authors: lnicola samuela

repo: rust-analyzer/rust-analyzer


text:
[feature request] support VSCode's native Testing UI

At present, VSCode's native Testing UI has been completed
can we use the testing ui api in ra?

Duplicate of #3601.

authors: findNextStep flodiebold


text:
rename symbol doesn't work across files in same crate

Should "rename symbol" and "find all references" work across a crate?
If I try renaming add in the following code, it works fine (also renaming the occurrence in main. if I put Q and it's impl into another file f.rs and use mod f to include it into main and use f::Q, now "rename symbol" doesn't find references to add in main.rs.
pub struct Q {
    i: u64,
}

impl Q {
    pub fn add(&self) -> u64 {
        self.i
    }
}

fn main() {
    let q = Q{i:1};
    q.add();
}

Yeah, I remember I added a FIXME in about this bug:
https://github.com/rust-analyzer/rust-analyzer/blob/8a73a8937d67db97072a11a40f0c3ed963e18e1b/crates/ra_ide_db/src/search.rs#L128-L131
I think it is not hard to fix which we could use the module visibility to check whether a sub-module needs to be included.

should be fixed now.

authors: ChrisJefferson edwin0cheng matklad


text:
Expand signature help

Signature help using call syntax with tuple structs and enum variants
Fixes #2102.

Currently this will try to complete:
struct S {
  foo: u32,
  bar: u32,
}

If you type in let s = S(<|> which is wrong.

Hm I think the way to solve the triggering on wrong type is to check if the StructKind is a tuple.

Okay I think this is ready for review.

bors r+

authors: kjeremy matklad


text:
Remove code duplication in tests

bors r+
🤖

bors r+

bors ping


text:
Go to Type Definition hover action.

This implementation supports things like dyn Trait<SomeType>, -> impl Trait, etc.

@flodiebold the walk implementation is ready.

Btw, is it worth to have two almost identical methods?
https://github.com/rust-analyzer/rust-analyzer/blob/36353bb1827dbd2efcde2d18c8598c4cc5e2e296/crates/ra_hir_ty/src/lib.rs#L808-L816
https://github.com/rust-analyzer/rust-analyzer/blob/36353bb1827dbd2efcde2d18c8598c4cc5e2e296/crates/ra_hir_ty/src/lib.rs#L871-L880
looks like dyn_trait() might be replaced with dyn_trait_ref().map(|it| it.trait_)

I think both are useful, though the implementation of dyn_trait could be replaced by that.

May I add this change to this PR, or it is better to do it later?

May I add this change to this PR, or it is better to do it later?

Since it's a very small change, I think either would be fine.

@flodiebold a new iteration is ready for review, though I did not manage to think out a test for Ty::associated_type_parent_trait, Ty::Projection arm.

authors: flodiebold vsrs


text:
Remove repetitive inlay hints

This removes/fixes unnecessary hints when the function name ends with the first param name, and increases readability.
Example:
s.strip_suffix(suffix: "Hello");

would now become
s.strip_suffix("Hello");

Happy Hacktoberfest! 🎉

I'm surprised at the amount of heuristics that we've amassed here.
In this specific case, though, I'm worried about something like fn max(x: u32, y: u32). Maybe we should check that the function name matches the parameter name, or ends in _ followed by it?

Oh wow, I didn't think about the fn max(x: u32, y: u32) scenario, I think the simplest way to fix this is to check if the fn_name ends with param_name prepended with _.
However, since single letter parameters aren't used unless its obvious, maybe we also just don't display single letter params?

I think the simplest way to fix this is to check if the fn_name ends with param_name prepended with _.

Maybe? There might be functions with the same name as their arguments, but I can't think of an example.

However, since single letter parameters aren't used unless its obvious, maybe we also just don't display single letter params?

Maybe? :laughing:

I am extremely sorry for accidentally closing it, I'm going to remove the unnecessary allocation now.

I think to be more flexible we should just expose a configuration for a list of regular expressions with call patterns like it is done in Intelij

I didn't mean to stall this PR, it is definitely useful now. My comment is more about possible far future changes =)

I think the heuristics work better because nobody has to compile a list of "obvious function signatures", not to mention that different people would have different notions of obviousness wrt. e.g. strncmp.

expose a configuration for a list of regular expressions with call patterns

Wouldn't that be best to do in an extension/client side?
To switch over you would need to:

[ ] Encourage every r-a extension out there to add this functionality
[ ] Slowly phase out heuristics outside the language server, making sure that users don't see any difference in user experience while this is happening.

CI is failing when testing proc_macro_srv, exiting with STATUS_ILLEGAL_INSTRUCTION. Any help?

Try to rebase over master.

Hey -- I'm sorry, I'm new to contributing. I tried to rebase as with the help provided by the check_merge_commits check, but now it's saying conflicting files, and if I try to resolve that on GitHub the check fails. Any guidance?

Hey, I've checked your branch and looks like you have the conflicts with master branch indeed:
both modified:   crates/ide/src/inlay_hints.rs
both modified:   crates/ide/src/references.rs
added by us:     crates/ide_db/src/apply_change.rs

Since you seem to have a single new commit in you branch, I suggest you to first pull the latest changes for the rust-analyzer's master branch and then rebase on top of it, fixing the conflicts for all 3 files.
Then you can force push into your remote branch to propagate the changes to the server.

Hi, Sorry @SomeoneToIgnore , I just did what you suggested, but then the build was failing as an import was missing, I just added that in the fix build commit, and now there's a conflict again. I'm really sorry, why does this happen?

There can be many reasons: you could  have not actually updated the master branch before resolving the conflicts, there can be new changes in that branch after you resolution, there can be something else I'm not aware of.
Anyway, you have to resolve those conflicts, so use the same approach again until they are gone.
If the merge conflicts do not disappear after a couple attempts, then you're doing something wrong and it'a a good time to stop and think thoroughly.

THere are also some hints here: https://github.com/rust-analyzer/rust-analyzer/blob/ef33953c70e361e8ab6b0bb6b0f5604e1f1c7f29/xtask/tests/tidy.rs#L67-L90

authors: SomeoneToIgnore Veetaha lnicola matklad matthewsanetra


text:
Are the derives on model types correct

I can't remember why I used the current set of derives e.g,
/// Models the result of JSON-RPC method `getbestblockhash`.
#[derive(Clone, Debug, PartialEq, Deserialize, Serialize)]
pub struct GetBestBlockHash(pub BlockHash);

And
/// Models the result of JSON-RPC method `getblock` with verbosity set to 0.
#[derive(Clone, Debug, PartialEq, Eq, Deserialize, Serialize)]
pub struct GetBlockVerbosityZero(pub Block);

Why no Ord/PartialOrd, is Eq correct?

We probably want Eq and PartialEq for tests, if nothing else. I think every RPC call returns structured data that has a sensible notion of equality.
Ordering/Hash is basically in case people want to put the types into a BTreeSet or something. But I don't think these types are intended to be stored, just to be immediately converted into something application-specific right?

I just remembered that the API guidelines say that data structures should eagerly implement these. I agree with your reasoning but I rekon we should throw them on.

While the API guidelines say we should eagerly impl them, Kix has argued elsewhere that it's potentially dangerous or confusing for types where there isn't a single obvious natural ordering.
Now, for these types, there is obviously no "natural ordering" that somebody would depend on; the types have no consensus meaning and live in no well-defined protocol; etc. So it's hard for me to imagine Ord causing anyone problems.
But I (weakly) vote that we leave them off for now and wait for somebody to file a bug.

Aight, that's good enough for me. I wasn't planning on 1.0-ing this caret anytime soon anyways. Thanks man.

authors: apoelstra tcharding

repo: rust-bitcoin/corepc


text:
Should Network::from_str accept different values?

At the moment we have "bitcoin" <-> Network::Bitcoin, then we have from_core_arg to accept "main" because it's the way bitcoin core refer to mainnet.
Lately I saw umbrel use "mainnet"
Is it acceptable to accept multiple values in the from_str impl ?
fn from_str(s: &str) -> Result<Self, Self::Err> {
        use Network::*;

        let network = match s {
            "bitcoin" | "mainnet" | "main" => Bitcoin,
            "testnet" | "test" => Testnet,
            "signet" => Signet,
            "regtest" => Regtest,
            _ => return Err(ParseNetworkError(s.to_owned())),
        };
        Ok(network)
    }

My two cents, I think it would be useful for the overall health of the library to tighten up at times like this and only do bare essentials in rust-bitcoin, pushing other stuff up to applications. Display/FromStr roundtripping is one place that we can have an implementation that is simple, does one thing, does it well, and does not need changing any time soon. On the other hand, allowing strings x, y, z to parse seems like something that is open to being patched a bunch.
A counter argument would be if we find duplicate code in libraries above us (miniscript, bdk, ldk) then perhaps it should be here in rust-bitcoin?

Agree with not over-complicating. I also don't think any downstream library should do this - only applications.

I understand and agree, thanks everyone

authors: Kixunil RCasatta tcharding

repo: rust-bitcoin/rust-bitcoin


text:
Make bitcoin-hashes optional, use traits/generics for hashes, so a library user can insert another implementation

When using this library in an embedded environment such as the BitBox02, binary size matters. In the BitBox02 firmware, there are already multiple implementations of various hashing functions. It would be great to use traits and generics where hashing is needed so we can reuse the existing implementations instead of pulling in another one, needlessly increasing the binary size.

I don't think you can make bitcoin_hashes optional since it defines newtypes, not just implements hashing functions. So if you want trait-based approach it needs to be in the bitcoin_hashes crate and should actually be discussed over there.

I definitely like this in principle but it does sound like a hard cross-crate coordination problem.
Even within this library we have multiple sha2 implementations since there is one hidden in side libsecp256k1.

So if you want trait-based approach it needs to be in the bitcoin_hashes crate and should actually be discussed over there.

Not necessarily. rust-bitcoin could define the traits it needs to provide further functionality and implement them upon certain dependency-features (like bitcoin_hashes) being activated.
pub trait HashToMessage {
	fn hash_sha256(data: &[u8]) -> Message;
}

#[cfg(feature = "bitcoin_hashes")]
pub struct BitcoinHashes;

#[cfg(feature = "bitcoin_hashes")]
impl HashToMessage for BitcoinHashes {
	// use bitcoin_hashes functions within trait implementations
}

Any function that needs hashing can the be parameterized over T: HashToMessage. With the traits being defined within rust-bitcoin, users could supply their own implementations if they wanted to.
This would essentially be a ports-and-adapter / onion / hexagonal architecture.

@thomaseizinger how would you combine this with newtypes found in bitcoin_hashes without changing them?

I wouldn't mind a feature flag that would pass through to sha2 if the user was willing to take the MSRV/stability hit.
Regarding generic traits, I wonder if we could just use constfns and associated constants to eliminate the code which is currently in a newtype macro.

@apoelstra mentioned we could solve this with a --cfg flag and calling to extern "C" functions:
https://github.com/rust-bitcoin/rust-bitcoin/issues/2881#issuecomment-2226942945
This is probably the most viable solution, as it doesn't pollute the whole code with generics.
extern "Rust" might be even better as Rust users then don't have to deal with the restrictions of C functions, while users who need C functions can still call to C.

The issue with that is we still haven't figured out what to do about state. The only thing that sees doable to me is to allocate on heap.

Yeah, for me that would be totally fine.

authors: Kixunil apoelstra benma thomaseizinger


text:
Use x86-linux-* for i585/i686 Rust targets on zig 0.11+

Fixes #120

We can drop support for older versions of zig so it's not a big deal.

repo: rust-cross/cargo-zigbuild


text:
Ease the readers into the Discovery book

resolves #195
The book has a great and slow introduction to embedded rust using the STM32F3, writing an hello world and then at chapter "Memory-Map registers" takes a hard turn, goes quite hardcore into hardware registers, using other hardware than the STM32F3. That can be quite intimidating for someone new to embedded world, who doesn't quite yet know what systick is, and may just want to blink some leds on his board.
For those Sunday tinkerers that got started with the rust-embedded book that do not yet have the technical chops to go through these details,  I tried to write an escape hatch to the more palatable Discovery Book.
I also updated a few bits of code that needed updates to match the latest tools update.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @therealprof (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

Thanks for this update, looks good!
I wonder if we should link to the newer https://crates.io/crates/stm32f3-discovery crate, which is recently updated, rather than the old f3 crate, which hasn't been updated in 2 years.

Thanks @adamgreig for the quick review!

I wonder if we should link to the newer https://crates.io/crates/stm32f3-discovery crate, which is recently updated, rather than the old f3 crate, which hasn't been updated in 2 years.

I agree, linking to a maintained crate sounds much better. I've updated my PR accordingly.
The Discovery book still uses that old f3 crate, let see how easily I can switch the crate to the stm32f3-discovery crate.

Ah boo, this is conflicting with https://github.com/rust-embedded/book/pull/248 which made the same use crate as _ changes. Please could you rebase or merge with master to resolve them?

Merge conflict resolved!

authors: adamgreig blelem rust-highfive

repo: rust-embedded/book


text:
cargo fuzz run all

I have a dozen of fuzz scripts.
It would be useful if cargo fuzz run could run them all. In the simplest case just for one second per target in a loop.
The use case is this: before release run the fuzzer for a night and check existing fuzz targets.

Fuzzing is supposed to be run indefinitely (and one second per fuzz isn't very useful at all). Different people would have different needs for this kind of thing (different timing requirements, etc). I recommend writing a script for this.

It should run indefinitely in a loop one second per fuzz target, preserving state.

Problem point is, if one have 20 fuzz target, it’s hard to automate running all of them for a night.

It should run indefinitely in a loop one second per fuzz target, preserving state.

Fuzzers don't really support this kind of thing. You can build it by manually sending SIGSTP, I don't really think it belongs in cargo fuzz.

authors: Manishearth stepancheg

repo: rust-fuzz/cargo-fuzz


text:
Add CPU and memory exhaustion in Lewton

Found by AFL and classified as hang

🎉

authors: Shnatsel frewsxcv

repo: rust-fuzz/trophy-case


text:
Radare2 bindings - Error { repr: Custom(Custom { kind: Other, error: StringError("Cannot find binary path") }) }

The file is https://github.com/radare/radare2/blob/master/libr/include/r_core.h This is my creduce predicate script:
CMD="bindgen --verbose /home/user/data/tmp/r_core.h -o /home/user/data/tmp/r2-api/r_core.rs -- -I/usr/local/include/libr"
OUT=$($CMD 3>&1 1>&2 2>&3)
EXPECT="Error { repr: Custom(Custom { kind: Other, error: StringError(\"Cannot find binary path\") }) }"

if [ "$OUT" == "$EXPECT" ]; then
	exit 0
else
	exit 1
fi

Here is the preprocessed file
__bindgen.i.gz
Sadly creduce cannot reduce this test because of the bug in the creduce itself - I sent them a report, they are checking it.
See also https://github.com/rust-lang-nursery/rust-bindgen/issues/1154

Ran into this myself. Could you add this to the README or the book, that in order to use the CLI, you need to install rustfmt-nightly (not the regular rustfmt)? A more descriptive error would be appreciated as well. Thanks.

Sorry for missing this one, yeah, this should be added to the docs... I can't do it right now but I'll put it in the queue. PRs welcome if you manage to get to it earlier of course. Sorry!

authors: XVilka emilio fschutt

repo: rust-lang-nursery/rust-bindgen


text:
Update appveyor image

Should reduce the flakiness of appveyor builds, as the new image disables windows defender.

This increases the total build time by ~8 minutes because msys2 is not preinstalled on the new image, but that should be fixed once it is merged to master, assuming the appveyor file cache works as expected.

@bors r+

:pushpin: Commit 826f439 has been approved by brson

:hourglass: Testing commit 826f439 with merge 75f222d...

:sunny: Test successful - status-appveyor, status-travis
Approved by: brson
Pushing 75f222d5747e26765778d5a71d8aab00792a1124 to master...

:eyes: Test was successful, but fast-forwarding failed: 422 Update is not a fast forward

authors: Diggsey bors brson


text:
apparent typo in macro for aarch64

so I don't know the first goddamn thing about ARM, but there's just something about this that don't look right:
https://github.com/rust-lang-nursery/stdsimd/blob/2f86c75a2479cf051b92fc98273daaf7f151e7a1/stdsimd/arch/detect/arch/aarch64.rs#L51-L54

Ah good catch, looks like a typo!

authors: ExpHP alexcrichton

repo: rust-lang-nursery/stdsimd


text:
Index out of bounds panic in UUID parser

Hello,
I managed to cause an index out of bounds panic in the parse_str function with the following input:
assert_eq!(Uuid::parse_str("231231212212423424324323477343246663"), Err(InvalidLength(36)));

The problem here is that the parser_str function checks for the input length (32 or 36), but does not have any "hyphen verification" in the 36 case. Because of this, the 33rd digit will cause the panic:
index out of bounds: the len is 16 but the index is 16', src/lib.rs:489
let mut buffer = [0u8; 16];
//...
buffer[(digit / 2) as usize] = acc;
//      ^^^^^^^^^
//         16

I tried to implement a quick fix in two different ways, but both have drawbacks.
I can count the hyphens when the input length is 36, or I can just check for the digit index variable in every iteration of the parser loop.
Which would be more efficient / cleaner solution? What kind of ParseError should I return in this case?

Ah I wouldn't worry too much about the performance of the parser here, I think it's not exactly heavily optimized today anyway. Also feel free to invent a new ParseError or slot this into an already existing varaint.

authors: MrAnno alexcrichton

repo: rust-lang-nursery/uuid


text:
error: Package does not have these features

This seems like a regression introduced recently... Specifying features through dependencies has worked for a long time, but is now broken:
$ cargo init testing &&
  cd testing &&
  echo -e 'byteorder = "*"' >> Cargo.toml &&
  cargo build --features byteorder/std
    Updating registry `https://github.com/rust-lang/crates.io-index`
error: Package `testing v0.1.0 (file:///tmp/testing/testing)` does not have these features: `byteorder/std`
$ cargo --version
cargo 0.13.0-nightly (d263690 2016-07-07)

The test case works as expected with cargo 0.12.0-nightly (50dad37 2016-06-21)

This was likely due to https://github.com/rust-lang/cargo/pull/2821.
In retrospect I'm not sure whether this should actually be allowed or not. On one hand you know when you're building a crate what the dependencies are, so it's not leaking information like before. On the other hand though you're just kinda poking around...
Out of curiosity, what was the use case of this?

cc @bennofs

The particular use case that triggered this was CI testing; ensuring that the package builds and tests against a dependency when certain features are enabled, even though that feature isn't explicitly a feature of the main crate.
Considering that dep/feature is valid in a Cargo.toml's [features] feature_name = ["dep/feature"], I think it makes sense that the same syntax be allowed on the CLI. Other crates in an upstream dependency chain may enable certain features on a crate that you depend on, so testing against that in CI can be useful in some cases.

That makes sense to me, and I agree that this is something we should fix. Thanks for the report!

@bennofs would you mind taking a look?

I agree with this report, specifying additional features for dependencies on the command line should definitely be allowed. I'll look it into this.

Thanks @bennofs!

@alexcrichton Do we want to support arbitrarily deep feature specifications on the command line? For example, --features "foo/bar/qux"? And what about allowing specifying features of dependencies that are not direct dependencies of the current package, but appear deeper in the dep chain? Say, if the current package does not depend on bar directly, but does so though transitive dependencies, should --features "bar/some-feat" be allowed?

@bennofs no I think we only want to allow foo/bar at most, and only when foo is a directly listed dependency of the current crate in Cargo.toml.

The particular use case that triggered this was CI testing; ensuring that the package builds and tests against a dependency when certain features are enabled, even though that feature isn't explicitly a feature of the main crate.

Enabling a feature on a library must never break any crate that depends on it. Doing so should be considered as a bug.
Your argument is more or less "I want to add tests in my library that tests whether my dependencies are buggy".
This is the most practical solution, but in my opinion it's the wrong way to fix the current situation.

authors: alexcrichton arcnmx bennofs tomaka

repo: rust-lang/cargo


text:
Experiment with giving higher priority to proc-macros and their dependencies

Currently, the dependency queue uses the number of transitive reverse dependencies to determine how to schedule jobs:
https://github.com/rust-lang/cargo/blob/5847787feff6db0eba19b0e433b2ae0baa132ca0/src/cargo/util/dependency_queue.rs#L34-L35
I think it would be a good idea to experiment with 'boosting' the priority of proc-macro crates and their dependencies. Unlike other crates, proc-macro crate building cannot be pipelined - since they are dynamically loaded into the compiler, any reverse-dependencies of proc-macro crates must wait for codegen, not just metadata.
By giving proc-macro crates a higher priority, we increase the likelihood that they will finish codegen by the time we are ready to start building their reverse dependencies. A similar argument applies to dependencies (not reverse-dependencies) of proc-macro crates: until https://github.com/rust-lang/rust/issues/64191 is implemented, we still need to wait for all a proc-macro crates's dependencies to codegen before we can start building the proc-macro crate.
Anecdotally, I've noticed that some proc-macro dependencies (e.g. syn) seem to take a significant fraction of the total build time. By starting to build these crates earlier, we may be able to reduce the impact of their build times on the total build time.
Concretely, I think we could add a flag -Z prioritize-proc-macros to Cargo. When enabled, we could either directly increase the priority entry by a certain factor, or just break ties in priority by proc-macro (or proc-macro-dependency) status.
By using -Z timings, we can gather data on the impact of this change on build times.

I think it would be great to experiment with this.  If you (or someone) wants to prototype it and demonstrate that it is measurably faster on some sampling of projects (and various -j values), I think it could be added without a -Z flag.

To add to this (while being unsure if this is even pursued at the moment): a similar logic applies to build scripts. For example, the now-stabilized --timings on a crate that uses openssl-sys/vendored (and thus has to build openssl from source to statically link it) shows that the critical dependency here (with some hops through openssl crates) is cc, which is built as the 7th crate total out of some hundred.
The build script only starts running about 15 seconds later, however, leading to a very undesirable tail of compiling the handful of crates that use it at then end. The build script takes a very long time to run, so all other crates are well done by that point - if the build script was started earlier, it could have run in parallel for those 15 seconds and the whole build would be faster.

authors: Aaron1011 domenicquirl ehuss


text:
Fix errors in cargo fetch usage guide

This is a recreation of #9785.

By the doc, cargo build will always access the network unless --frozen or --offline is present. Instead, cargo fetch will fetch dependencies, and which will allow running subsequent Cargo commands in offline mode.

A general template was used for --target that is not appropriate for cargo fetch.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
Please see the contribution instructions for more information.

Would it be better to use #if helper to avoid duplicate? Like #9189 did.

I agree with @weihanglo that using #if to keep the --target docs in one spot is probably a good idea, otherwise seems good to me though!

So, should I add 3 flags for:
1. Change "The default is the host platform." to "The default is for {{actionverb}} for all architecture."

2. Add a sentence "Use the value `all` to include _all_ targets."

3. Hide the paragraph "Note that specifying this flag makes Cargo run in a different mode..."

respectively?

I feel like cargo-tree diverges from the common one to some extent, so I vote for keeping cargo-tree as is.

@bors: r=weihanglo

:pushpin: Commit 08734a664362b9ea39ba7d15ef340eff25749af4 has been approved by weihanglo

:hourglass: Testing commit 08734a664362b9ea39ba7d15ef340eff25749af4 with merge c974e3528c9a81bc900a23114302538de293f2be...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing c974e3528c9a81bc900a23114302538de293f2be to master...

authors: alexcrichton bors piggynl rust-highfive weihanglo


text:
Don't use for loop on an Option

This PR removes a single for loop over Option, replacing it with an if let to improve code clarity. This currently blocks https://github.com/rust-lang/rust/pull/99696 that adds a lint against this pattern.

r? @epage
(rust-highfive has picked a reviewer for you, use r? to override)

@bors r+

:pushpin: Commit d8f8e8fa58e6a09ee72c8978c89115f178be44e6 has been approved by epage
It is now in the queue for this repository.

:hourglass: Testing commit d8f8e8fa58e6a09ee72c8978c89115f178be44e6 with merge 0eac8be633e5fdbd93ee31a8ffaada74e637a046...

:sunny: Test successful - checks-actions
Approved by: epage
Pushing 0eac8be633e5fdbd93ee31a8ffaada74e637a046 to master...

authors: WaffleLapkin bors epage rust-highfive


text:
Disallow RUSTUP_HOME in the [env] table.

This adds a check to prevent RUSTUP_HOME from being set in the [env] config table under the same reasoning as was done in #11590. Cargo will likely behave incorrectly if this key is set in the config since it will not change the home used by the outer cargo itself.
This is a breaking change, though I think it is unlikely to be used in practice. When cargo is executed via a rustup proxy, the proxy sets RUSTUP_HOME which overrides the [env] table entry. It may be feasible that someone is invoking cargo directly without the rustup wrappers, and then using this to steer the rustc invocations to a different rustup location, but I'm not sure that's a use case we need to be supporting.
This is being added as a further assurance for #11590 to make sure the environment is configured as expected.
We could potentially add other Rustup env vars to reject, but I'm not sure I want to delve into analyzing all the possible reasons or interactions for each one.

Sure! Added a comment.

@bors r+

:pushpin: Commit 2fce10d0217a4182fafe37dd95484f04833339bc has been approved by weihanglo
It is now in the queue for this repository.

:hourglass: Testing commit 2fce10d0217a4182fafe37dd95484f04833339bc with merge 1490d10bd6d61426853ffb73fef631576aff0e78...

:sunny: Test successful - checks-actions
Approved by: weihanglo
Pushing 1490d10bd6d61426853ffb73fef631576aff0e78 to master...

@ehuss RUSTUP_TOOLCHAIN seems like one to ban too ? (That was actually the one I expected from #11917 :P.

Sure, I went ahead and posted #12107.

authors: bors ehuss rbtcollins weihanglo


text:
respect configure options for paths

This change causes the installed paths of some files to respect the options passed to configure. It additionally adds another option and removes an unused option.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

So I will readily admit I haven't actually build a cargo with these changes yet. I'm in the process of attempting to package it up for Gentoo and I'm looking at the ebuilds that Gentoo users have put together and the sed hacks or patches they're doing to the cargo Makefiles and this looks like it should handle everything and also provide some clean ups. I figured I'd post these to start a discussion.

Ok so needed to have the --infodir parameter for the configure script because a number of distros set that when calling ./configure so I dropped the patch that removed it. I've now used these patches to build cargo for Gentoo and they'll be used in the official package of 0.10.0 so it would be nice for them to be considered for upstream for future releases.

@bors: r+
Thanks!

:pushpin: Commit 68792d7 has been approved by alexcrichton

:hourglass: Testing commit 68792d7 with merge 7e34d2c...

:sunny: Test successful - cargo-cross-linux, cargo-linux-32, cargo-linux-64, cargo-mac-32, cargo-mac-64, cargo-win-gnu-32, cargo-win-gnu-64, cargo-win-msvc-32, cargo-win-msvc-64

authors: alexcrichton bors cardoe rust-highfive


text:
Add required-features to SerializedTarget

This will add it to cargo metadata output and will make it
possible to enable features needed to build specific target.

Thanks for the pull request, and welcome! The Rust team is excited to review your changes, and you should hear from @alexcrichton (or someone else) soon.
If any changes to this PR are deemed necessary, please add them as extra commits. This ensures that the reviewer can see what has changed since they last reviewed the code. Due to the way GitHub handles out-of-date commits, this should also make it reasonably obvious what issues have or haven't been addressed. Large or tricky changes may require several passes of review and changes.
Please see the contribution instructions for more information.

One of the use cases is https://github.com/flycheck/flycheck-rust/issues/68. Let me know, if it's not an ideal solution!

@alexcrichton hi Alex! Does it look like smth we can merge? Thanks!

Looks good to me, thanks! Could this also be renamed to required-features in the JSON?

@alexcrichton Sure! I've pushed additional commit -- is it ok, or should I squash both commits together?
Field attribute is now longer than 80 chars, but it seems cargo codebase already has attributes like this. Happy to fix it though.
Thanks for you time!

@bors: r+
Looks good to me, thanks!

:pushpin: Commit 349d264a5678a93e27d514b149503c3ed278858a has been approved by alexcrichton

@bors: retry

@bors: r+

:bulb: This pull request was already approved, no need to approve it again.

:pushpin: Commit 349d264a5678a93e27d514b149503c3ed278858a has been approved by alexcrichton

@bors: r+

:pushpin: Commit 9c38505ab768fc36f476286bfc25c90c474bf2c4 has been approved by alexcrichton

:hourglass: Testing commit 9c38505ab768fc36f476286bfc25c90c474bf2c4 with merge 0dc7f69d0f1065b4a901b195acaa50795900d438...

:broken_heart: Test failed - status-appveyor

failures:
---- build_script::switch_features_rerun stdout ----
running `C:\projects\cargo\target\debug\cargo.exe run -v --features=foo`
running `C:\projects\cargo\target\debug\cargo.exe run -v`
running `C:\projects\cargo\target\debug\cargo.exe run -v --features=foo`
thread 'build_script::switch_features_rerun' panicked at '
Expected: execs
    but: exited with exit code: 101
--- stdout
--- stderr
       Fresh foo v0.0.1 (file:///C:/projects/cargo/target/cit/t353/foo)
error: failed to link or copy `C:\projects\cargo\target\cit\t353\foo\target\debug\deps\foo-86673955dda3b155.exe` to `C:\projects\cargo\target\cit\t353\foo\target\debug\foo.exe`
Caused by:
  Access is denied. (os error 5)
', tests\testsuite\support\hamcrest.rs:13:9
note: Run with `RUST_BACKTRACE=1` for a backtrace.

hm, environment error?

Looks like it would need the same treatment as changing_bin_features_caches_targets.  It's unfortunate, and a little awkward.  Personally I would prefer to add retry logic to cargo itself, since there is likely other tests that are vulnerable, and will likely be new ones in the future.

:hourglass: Testing commit 9c38505ab768fc36f476286bfc25c90c474bf2c4 with merge c0ec76f336dbeaa819e32d21eb0c541ef875cf69...

@alexcrichton any idea when this feature will be available in nightly?
I've updated today and cargo is still from 0ec7281b9 2018-08-20.

@little-arhat update PR posted at https://github.com/rust-lang/rust/pull/53935, once it gets through it should be in the next nightly.

authors: alexcrichton bors ehuss little-arhat rust-highfive


text:
azure_mgmt_sql 8 GB

Crate name
azure_mgmt_sql
Build failure link
https://docs.rs/crate/azure_mgmt_sql/0.1.0/builds/499552
Requested RAM limit
8 GB
Requested timeout
No response
Requested number of targets
No response
Why your crate needs the resource increases
Build failed at 3 GB.

@cataggar is this still needed?

authors: cataggar syphar


text:
Utilize multi-threaded tokio runtime for move-to-s3

The current thread runtime was previously used but there was no reason
to do so. This utilizes the default runtime.

I seem to recall getting a compilation error last time I tried this, but I must have changed something since then.

Thanks!

authors: Mark-Simulacrum onur


text:
Provide a way to run a test that needs an external dependency

It would be pretty neat if we had a way to have ./miri run work on tests that need external dependencies -- right now, these are a bit annoying to debug (and for tests that call into libc, this is a regression compared to the state before we added support for test dependencies). This should probably be a separate ./miri run-dep or so, and as much as possible it should share the test-dependencies infrastructure with ui_test/compiletest.

I just ran into this yesterday, where I wanted to just run some files as a quick test, and the libc dependency became a problem. So it'd be great to have some sort of solution here.

This should work for files outside of the test folder. It should also not compare the output to any stderr/stdout files, nor write any stderr/stdout files itself. It should ignore magic test comments. It should just run the file and not have any expectations about the exit code.
Using ./miri test kind of works but it is a crude work-around -- there's a lot of junk in the output, one has to put the file into a particular folder, one cannot overwrite the magic comments.

This would also have been really useful to debug https://github.com/rust-lang/futures-rs/issues/2669.
Judging from https://github.com/oli-obk/ui_test/pull/11 being closed, looks like this "just" needs some wiring up in Miri?

Ah yes, I forgot about this one after implementing the ui_test side of things

I won't get to this this year and January will most likely be spent catching up on other things. Will pick it up again after that.

This works great, thanks!
However printing the entire invocation makes this very verbose and makes it easy to miss the actual program output. Maybe this should be hidden, at least by default?

We can just stop emitting that. I thought it may be practical if something goes wrong but I don't see a good way to hide it conditionally. We could make it a flag before the file name I guess?

If something goes wrong I'd probably rather debug ./miri run and cargo miri run than the complicated dep setup.
So yeah for now I'd vote for not emitting that.

authors: RalfJung oli-obk

repo: rust-lang/miri


text:
Fix use of deprecated check_no_isolation in posix fs shims

Update posix fs shims to use new API reject_in_isolation, which
allows rejection with error code instead of always forcing abort.
Error code chosen for each op is the most appropriate one from the
list in corresponding syscall's manual.
Updated helper APIs to not use quotes (`) around input name while
preparing the message. This allows callers to pass multi-word string
like -- "`read` from stdin".

Could you also add a test that exercises these functions? Something like tests/run-pass/current_dir_with_isolation.rs (but it doesn't need to do that loop, calling the function a single time suffices).

@atsmtat it's been a while since we heard from you, what is the status of this PR?
I'm afraid there will probably be a lot of conflicts with https://github.com/rust-lang/miri/pull/1851.

@atsmtat it's been a while since we heard from you, what is the status of this PR?
I'm afraid there will probably be a lot of conflicts with #1851.

Sorry for the late reply. I got busy. I addressed your comments in a new commit I just pushed.
I can wait for your PR to merge and then rebase and resolve conflicts.

@atsmtat it's been a while since we heard from you, what is the status of this PR?
I'm afraid there will probably be a lot of conflicts with #1851.

Sorry for the late reply. I got busy. I addressed your comments in a new commit I just pushed.
I can wait for your PR to merge and then rebase and resolve conflicts.

Just noticed that your PR is merged. I rebased my branch and resolved conflicts, wasn't too bad 🙂

Awesome, thanks a lot. :)
@bors r+

:pushpin: Commit 20d0f2ee2607a1eecf095223250ca07307b9dbb0 has been approved by RalfJung

:hourglass: Testing commit 20d0f2ee2607a1eecf095223250ca07307b9dbb0 with merge 1677946d74e411b497cf06e69749e7e1b15afa09...

:sunny: Test successful - checks-actions
Approved by: RalfJung
Pushing 1677946d74e411b497cf06e69749e7e1b15afa09 to master...

authors: RalfJung atsmtat bors


text:
Add backreferences and nested subpatterns

Named and numbered backreferences such as ^(a(?1)?b)$ are common in PCRE implementations, and would be very useful here.

Is it possible to do them without backtracking (and the associated risk of exponential time blow-up)?

@alexchandel Per the docs:

Notably, backreferences and arbitrary lookahead/lookbehind assertions are not provided. In return, regular expression searching provided by this package has excellent worst case performance.

Russ Cox's regex articles are useful here: https://swtch.com/~rsc/regexp/
My suggestion is that if you want PCRE's extra features, then you should probably just use PCRE. rust-pcre exists, although I'm not sure if it's being maintained.

Is it possible to do them without backtracking

I would imagine so.

Per the docs

That's my point. The issue is that they aren't implemented. Please reopen.

then you should probably just use PCRE

My intention is specifically not to link against PCRE, but use a rust implementation. Hence this repository. Backreferences fit neatly within the concept of Pay for what you use. A regex that doesn't compute the locations of subexpressions doesn't pay for the cost, and regexes that don't use backreferences wouldn't pay for them either.

Imagining isn't quite enough in this case. All evidence I can find (e.g. the more advanced re2 library not implementing them) strongly suggests it unfortunately isn't possible.

authors: BurntSushi alexchandel huonw

repo: rust-lang/regex


text:
Add bound contraint for trait associated const

I recently post on rust forum about how to write constraint on associated const, and find out that there is no direct way to achieve this (even no RFC?).
For example, I have the following trait:
trait Foo {
    const FOO: usize;
}

And I want to have constraint for that like
fn foo<T: Foo>() where <T as Foo>::FOO < 4 {}

impl<T: Foo> SomeTrait for T where <T as Foo>::FOO.is_power_of_two() {}

const fn something_about_foo(foo: usize) -> bool { /* ... */ }
struct Bar<T: Foo> where something_about_foo(<T as Foo>::FOO) {}

Since we already have various mature bound constraint on associated type, there should be some bound constraint on associated const as well.
I also wonder how to design the syntax of associated const bound, since allowing any const expression in where clause may lead to hard parsing. Maybe a brace should be required?

This is already possible in Nightly Rust today, by using const_generics and const_evaluatable_checked features as well as something analogous to C++'s std::conditional:
#![feature(const_generics)]
#![feature(const_evaluatable_checked)]

trait Satisfied {}

struct Cond<const PRED: bool>;

impl Satisfied for Cond<true> {}

trait Foo {
    const FOO: usize;
}

struct TestFoo;

impl Foo for TestFoo {
    const FOO: usize = 3;
}

fn foo<T: Foo>() where Cond<{<T as Foo>::FOO < 4}>: Satisfied {}

fn main() {
    foo::<TestFoo>();
}

But I agree there should be a more terse syntax for const constraints (a.k.a. refinements).

See also #3162.

authors: Evian-Zhang Xiretza lqf96

repo: rust-lang/rfcs