text:
Added PRHelpMessage to command execution in pr_agent.py

Type
enhancement

Description

Added a new PRHelpMessage class to provide automated help messages for PRs.
Integrated PRHelpMessage into the PR Agent's command execution flow, allowing users to request help directly in PR comments.
The help message includes links to documentation for various PR Agent tools, enhancing user guidance and support.

Changes walkthrough
Relevant filesEnhancement

pr_agent.pyIntegration of PRHelpMessage Command in Agent

pr_agent/agent/pr_agent.py
Imported PRHelpMessage from pr_agent.tools.  Added help command to the command execution mapping.

+2/-0

pr_help_message.pyImplementation of PRHelpMessage for Automated Help Responses

pr_agent/tools/pr_help_message.py
Implemented PRHelpMessage class for generating help messages in PRs.  The class fetches settings, prepares a help message, and publishes it as a PR comment.  Added documentation links for various PR Agent tools in the help message.

+62/-0

 ✨ Usage guide:
Overview:
The describe tool scans the PR code changes, and generates a description for the PR - title, type, summary, walkthrough and labels. The tool can be triggered automatically every time a new PR is opened, or can be invoked manually by commenting on a PR.
When commenting, to edit configurations related to the describe tool (pr_description section), use the following template:
/describe --pr_description.some_config1=... --pr_description.some_config2=...

With a configuration file, use the following template:
[pr_description]
some_config1=...
some_config2=...

  Enabling\disabling automation

When you first install the app, the default mode for the describe tool is:

pr_commands = ["/describe --pr_description.add_original_user_description=true"
                         "--pr_description.keep_original_user_title=true", ...]

meaning the describe tool will run automatically on every PR, will keep the original title, and will add the original user description above the generated description.

Markers are an alternative way to control the generated description, to give maximal control to the user. If you set:

pr_commands = ["/describe --pr_description.use_description_markers=true", ...]

the tool will replace every marker of the form pr_agent:marker_name in the PR description with the relevant content, where marker_name is one of the following:

type: the PR type.
summary: the PR summary.
walkthrough: the PR walkthrough.

Note that when markers are enabled, if the original PR description does not contain any markers, the tool will not alter the description at all.

  Custom labels
The default labels of the describe tool are quite generic: [Bug fix, Tests, Enhancement, Documentation, Other].
If you specify custom labels in the repo's labels page or via configuration file, you can get tailored labels for your use cases.
Examples for custom labels:

Main topic:performance - pr_agent:The main topic of this PR is performance
New endpoint - pr_agent:A new endpoint was added in this PR
SQL query - pr_agent:A new SQL query was added in this PR
Dockerfile changes - pr_agent:The PR contains changes in the Dockerfile
...

The list above is eclectic, and aims to give an idea of different possibilities. Define custom labels that are relevant for your repo and use cases.
Note that Labels are not mutually exclusive, so you can add multiple label categories.
Make sure to provide proper title, and a detailed and well-phrased description for each label, so the tool will know when to suggest it.

  Inline File Walkthrough 💎
For enhanced user experience, the describe tool can add file summaries directly to the "Files changed" tab in the PR page.
This will enable you to quickly understand the changes in each file, while reviewing the code changes (diffs).
To enable inline file summary, set pr_description.inline_file_summary in the configuration file, possible values are:

'table': File changes walkthrough table will be displayed on the top of the "Files changed" tab, in addition to the "Conversation" tab.
true: A collapsable file comment with changes title and a changes summary for each file in the PR.
false (default): File changes walkthrough will be added only to the "Conversation" tab.

  Utilizing extra instructions
The describe tool can be configured with extra instructions, to guide the model to a feedback tailored to the needs of your project.
Be specific, clear, and concise in the instructions. With extra instructions, you are the prompter. Notice that the general structure of the description is fixed, and cannot be changed. Extra instructions can change the content or style of each sub-section of the PR description.
Examples for extra instructions:
[pr_description]
extra_instructions="""
- The PR title should be in the format: '<PR type>: <title>'
- The title should be short and concise (up to 10 words)
- ...
"""

Use triple quotes to write multi-line instructions. Use bullet points to make the instructions more readable.

  More PR-Agent commands

To invoke the PR-Agent, add a comment using one of the following commands:

/review: Request a review of your Pull Request.
/describe: Update the PR title and description based on the contents of the PR.
/improve [--extended]: Suggest code improvements. Extended mode provides a higher quality feedback.
/ask <QUESTION>: Ask a question about the PR.
/update_changelog: Update the changelog based on the PR's contents.
/add_docs 💎: Generate docstring for new components introduced in the PR.
/generate_labels 💎: Generate labels for the PR based on the PR's contents.
/analyze 💎: Automatically analyzes the PR, and presents changes walkthrough for each component.

See the tools guide for more details.
To list the possible configuration parameters, add a /config comment.

See the describe usage page for a comprehensive guide on using this tool.

/review

/review

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:
- [DESCRIBE](https://github.com/Codium-ai/pr-agent/tree/main/docs/DESCRIBE.md)
- [REVIEW](https://github.com/Codium-ai/pr-agent/tree/main/docs/REVIEW.md)
- [IMPROVE](https://github.com/Codium-ai/pr-agent/tree/main/docs/IMPROVE.md)
- [ASK](https://github.com/Codium-ai/pr-agent/tree/main/docs/ASK.md)
- [SIMILAR_ISSUE](https://github.com/Codium-ai/pr-agent/tree/main/docs/SIMILAR_ISSUE.md)
- [UPDATE CHANGELOG](https://github.com/Codium-ai/pr-agent/tree/main/docs/UPDATE_CHANGELOG.md)
- [ADD DOCUMENTATION](https://github.com/Codium-ai/pr-agent/tree/main/docs/ADD_DOCUMENTATION.md)
- [GENERATE CUSTOM LABELS](https://github.com/Codium-ai/pr-agent/tree/main/docs/GENERATE_CUSTOM_LABELS.md)
- [Analyze](https://github.com/Codium-ai/pr-agent/tree/main/docs/Analyze.md)
- [Test](https://github.com/Codium-ai/pr-agent/tree/main/docs/TEST.md)
- [CI Feedback](https://github.com/Codium-ai/pr-agent/tree/main/docs/CI_FEEDBACK.md)

Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:
- [DESCRIBE](https://github.com/Codium-ai/pr-agent/tree/main/docs/DESCRIBE.md)
- [REVIEW](https://github.com/Codium-ai/pr-agent/tree/main/docs/REVIEW.md)
- [IMPROVE](https://github.com/Codium-ai/pr-agent/tree/main/docs/IMPROVE.md)
- [ASK](https://github.com/Codium-ai/pr-agent/tree/main/docs/ASK.md)
- [SIMILAR_ISSUE](https://github.com/Codium-ai/pr-agent/tree/main/docs/SIMILAR_ISSUE.md)
- [UPDATE CHANGELOG](https://github.com/Codium-ai/pr-agent/tree/main/docs/UPDATE_CHANGELOG.md)
- [ADD DOCUMENTATION](https://github.com/Codium-ai/pr-agent/tree/main/docs/ADD_DOCUMENTATION.md)
- [GENERATE CUSTOM LABELS](https://github.com/Codium-ai/pr-agent/tree/main/docs/GENERATE_CUSTOM_LABELS.md)
- [Analyze](https://github.com/Codium-ai/pr-agent/tree/main/docs/Analyze.md)
- [Test](https://github.com/Codium-ai/pr-agent/tree/main/docs/TEST.md)
- [CI Feedback](https://github.com/Codium-ai/pr-agent/tree/main/docs/CI_FEEDBACK.md)

Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:

DESCRIBE
REVIEW
IMPROVE
ASK
SIMILAR_ISSUE
UPDATE CHANGELOG
ADD DOCUMENTATION
GENERATE CUSTOM LABELS
Analyze
Test
CI Feedback

Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:

✨ DESCRIBE
✨ REVIEW
✨ IMPROVE
✨ ASK
✨ SIMILAR_ISSUE
✨ UPDATE CHANGELOG
✨ ADD DOCUMENTATION
GENERATE CUSTOM LABELS
Analyze
Test
CI Feedback

Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:

DESCRIBE
REVIEW
IMPROVE
ASK
SIMILAR_ISSUE
UPDATE CHANGELOG
ADD DOCUMENTATION
GENERATE CUSTOM LABELS
Analyze
Test
CI Feedback

Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:
ToolDescription[DESCRIBE]({base_path}/DESCRIBE.md)Generate PR description - title, type, summary, code walkthrough and labels[REVIEW]({base_path}/REVIEW.md)Adjustable feedback about the PR, possible issues, security concerns, review effort and more[IMPROVE]({base_path}/IMPROVE.md)Code suggestions for improving the PR.[ASK]({base_path}/ASK.md)Answering free-text questions about the PR.[SIMILAR_ISSUE]({base_path}/SIMILAR_ISSUE.md)Automatically retrieves and presents similar issues.[UPDATE CHANGELOG]({base_path}/UPDATE_CHANGELOG.md)Automatically updates the changelog.[ADD DOCUMENTATION]({base_path}/ADD_DOCUMENTATION.md)Automatically adds documentation to methods/functions/classes that changed in the PR.[GENERATE CUSTOM LABELS]({base_path}/GENERATE_CUSTOM_LABELS.md)Automatically generates custom labels for the PR.[Analyze]({base_path}/Analyze.md)Identify which code components changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.[Test]({base_path}/TEST.md)Generates unit tests for a selected component, based on the PR code change.[CI Feedback]({base_path}/CI_FEEDBACK.md)Generates feedback and analysis for a failed CI job.[Custom Suggestions]({base_path}/CUSTOM_SUGGESTIONS.md)Generates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerate PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR_ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
AnalyzeIdentify which code components changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TestGenerates unit tests for a selected component, based on the PR code change.
CI FeedbackGenerates feedback and analysis for a failed CI job.
Custom SuggestionsGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here are the tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerate PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentify which code components changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Intro
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVECode suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each command be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVE
Code suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription
DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels
REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more
IMPROVE
Code suggestions for improving the PR.
ASKAnswering free-text questions about the PR.
SIMILAR ISSUEAutomatically retrieves and presents similar issues.
UPDATE CHANGELOGAutomatically updates the changelog.
ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.
GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.
ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.
TESTGenerates unit tests for a selected component, based on the PR code change.
CI FEEDBACKGenerates feedback and analysis for a failed CI job.
CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription

[DESCRIBE](https://github.com/Codium-ai/pr-agent/tree/main/docs/DESCRIBE.md)Generates PR description - title, type, summary, code walkthrough and labels

[REVIEW](https://github.com/Codium-ai/pr-agent/tree/main/docs/REVIEW.md)Adjustable feedback about the PR, possible issues, security concerns, review effort and more

[IMPROVE](https://github.com/Codium-ai/pr-agent/tree/main/docs/IMPROVE.md)
Code suggestions for improving the PR.

[ASK](https://github.com/Codium-ai/pr-agent/tree/main/docs/ASK.md)Answering free-text questions about the PR.

[SIMILAR ISSUE](https://github.com/Codium-ai/pr-agent/tree/main/docs/SIMILAR_ISSUE.md)Automatically retrieves and presents similar issues.

[UPDATE CHANGELOG](https://github.com/Codium-ai/pr-agent/tree/main/docs/UPDATE_CHANGELOG.md)Automatically updates the changelog.

[ADD DOCUMENTATION](https://github.com/Codium-ai/pr-agent/tree/main/docs/ADD_DOCUMENTATION.md)Automatically adds documentation to methods/functions/classes that changed in the PR.

[GENERATE CUSTOM LABELS](https://github.com/Codium-ai/pr-agent/tree/main/docs/GENERATE_CUSTOM_LABELS.md)Automatically generates custom labels for the PR.

[ANALYZE](https://github.com/Codium-ai/pr-agent/tree/main/docs/Analyze.md)Identifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.

[TEST](https://github.com/Codium-ai/pr-agent/tree/main/docs/TEST.md)Generates unit tests for a selected component, based on the PR code change.

[CI FEEDBACK](https://github.com/Codium-ai/pr-agent/tree/main/docs/CI_FEEDBACK.md)Generates feedback and analysis for a failed CI job.

[CUSTOM SUGGESTIONS](https://github.com/Codium-ai/pr-agent/tree/main/docs/CUSTOM_SUGGESTIONS.md)Generates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription

DESCRIBE
Generates PR description - title, type, summary, code walkthrough and labels

[REVIEW](https://github.com/Codium-ai/pr-agent/tree/main/docs/REVIEW.md)
Adjustable feedback about the PR, possible issues, security concerns, review effort and more

[IMPROVE](https://github.com/Codium-ai/pr-agent/tree/main/docs/IMPROVE.md)
Code suggestions for improving the PR.

[ASK](https://github.com/Codium-ai/pr-agent/tree/main/docs/ASK.md)
Answering free-text questions about the PR.

[SIMILAR ISSUE](https://github.com/Codium-ai/pr-agent/tree/main/docs/SIMILAR_ISSUE.md)Automatically retrieves and presents similar issues.

[UPDATE CHANGELOG](https://github.com/Codium-ai/pr-agent/tree/main/docs/UPDATE_CHANGELOG.md)Automatically updates the changelog.

[ADD DOCUMENTATION](https://github.com/Codium-ai/pr-agent/tree/main/docs/ADD_DOCUMENTATION.md)Automatically adds documentation to methods/functions/classes that changed in the PR.

[GENERATE CUSTOM LABELS](https://github.com/Codium-ai/pr-agent/tree/main/docs/GENERATE_CUSTOM_LABELS.md)Automatically generates custom labels for the PR.

[ANALYZE](https://github.com/Codium-ai/pr-agent/tree/main/docs/Analyze.md)Identifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.

[TEST](https://github.com/Codium-ai/pr-agent/tree/main/docs/TEST.md)Generates unit tests for a selected component, based on the PR code change.

[CI FEEDBACK](https://github.com/Codium-ai/pr-agent/tree/main/docs/CI_FEEDBACK.md)Generates feedback and analysis for a failed CI job.

[CUSTOM SUGGESTIONS](https://github.com/Codium-ai/pr-agent/tree/main/docs/CUSTOM_SUGGESTIONS.md)Generates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription

DESCRIBE
Generates PR description - title, type, summary, code walkthrough and labels

REVIEW
Adjustable feedback about the PR, possible issues, security concerns, review effort and more

IMPROVE
Code suggestions for improving the PR.

ASK
Answering free-text questions about the PR.

SIMILAR ISSUEAutomatically retrieves and presents similar issues.

UPDATE CHANGELOGAutomatically updates the changelog.

ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.

GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.

ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.

TESTGenerates unit tests for a selected component, based on the PR code change.

CI FEEDBACKGenerates feedback and analysis for a failed CI job.

CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription

DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels

REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more

IMPROVECode suggestions for improving the PR.

ASKAnswering free-text questions about the PR.

SIMILAR ISSUEAutomatically retrieves and presents similar issues.

UPDATE CHANGELOGAutomatically updates the changelog.

ADD DOCUMENTATIONAutomatically adds documentation to methods/functions/classes that changed in the PR.

GENERATE CUSTOM LABELSAutomatically generates custom labels for the PR.

ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.

TESTGenerates unit tests for a selected component, based on the PR code change.

CI FEEDBACKGenerates feedback and analysis for a failed CI job.

CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be applied automatically when a new PR is opened, or invoked manually by commenting on a PR.

PR Agent Walkthrough
🤖 Welcome to the PR Agent, an AI-powered tool for automated pull request analysis, feedback, suggestions and more.
Here is a list of tools you can use to interact with the PR Agent:
ToolDescription

DESCRIBEGenerates PR description - title, type, summary, code walkthrough and labels

REVIEWAdjustable feedback about the PR, possible issues, security concerns, review effort and more

IMPROVECode suggestions for improving the PR.

ASKAnswering free-text questions about the PR.

SIMILAR ISSUEAutomatically retrieves and presents similar issues.

UPDATE CHANGELOGAutomatically updates the changelog.

ADD DOCUMENTATIONGenerates documentation to methods/functions/classes that changed in the PR.

GENERATE CUSTOM LABELSGenerates custom labels for the PR, based on specific guidelines defined by the user

ANALYZEIdentifies code components that changed in the PR, and enable to interactively generate tests, docs, and code improvements for each component.

TESTGenerates unit tests for a selected component, based on the PR code change.

CI FEEDBACKGenerates feedback and analysis for a failed CI job.

CUSTOM SUGGESTIONSGenerates custom suggestions for improving the PR code, based on specific guidelines defined by the user.
Note that each tool be invoked automatically when a new PR is opened, or called manually by commenting on a PR.


text:
fix: resolved bug where some old replaceable events were not being de…

…leted

Pull Request Test Coverage Report for Build 7449270765

0 of 0   changed or added relevant lines in 0 files are covered.
No unchanged relevant lines lost coverage.
Overall coverage decreased (-17.3%) to 76.056%

Totals

Change from base Build 7386731806:
-17.3%

Covered Lines:
398

Relevant Lines:
515

💛  - Coveralls

source: gharchive/pull-request

authors: CodyTseng coveralls

repo: CodyTseng/nostr-relay-nestjs

license: MIT

license_type: permissive

license_source: github-api


text:
Release as ROS Debian packages blocked

by https://github.com/CogRob/catkin_grpc/issues/17

grpc is fixed, submitted a new PR: https://github.com/ros/rosdistro/pull/16033
It has already been merged. Let's see if it will pass this time.

Build succeed. http://build.ros.org/job/Ibin_uT64__gcloud_speech__ubuntu_trusty_amd64__binary/

source: gharchive/issue


text:
Analyse does not exit in GUI

Describe the bug
The Analyse phase runs for ever, not exiting.
To Reproduce
Unknown
Expected behavior
It should only take a few seconds!

Debug version of GUI created

1.0.4d. Created to further analyse.

Fixed in 1.1.
Two issues were the cause.

Duplicate members handling going wrong.
inactive member handling, if inactive without a last modified date.

repo: ColinRobbins/scm-helper


text:
fix 206 range request

Second part of the fix to https://github.com/CollectionFS/Meteor-CollectionFS/issues/502

I have updated the PR. The previous version was not playing well with request errors.

@cherbst, cool looks good! Can you change it to merge into devel instead of master? We move to master once we release it.

Do I miss something here? There is no branch devel

No, my bad. I thought this was the main CFS repo.

authors: aldeed cherbst

repo: CollectionFS/Meteor-http-methods

license: mit

license_source: bigquery


text:
Make Announcement Command

Create an embedded message
Opens up a modal for message text

There isn't really a big need for this, the announcement channel and events tab pretty much do this already.

authors: JaydenPahukula keenanbuckley

repo: ColoradoSchoolOfMines/acm-discord-bot


text:
🛑 Ablaze Blog is down

In 6c6a850, Ablaze Blog (https://blog.ablaze.one) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ablaze Blog is back up in eefb50e.

repo: Comamoca/status


text:
Empty TLV Suppression

Suppress empty TLVs during decompile.

Feature Implemented in branch Issues_4.
Ready to be committed to Master and build next jar version

repo: Comcast/Oscar


text:
Add md5 calculation, storage and verification

We want to calculate the MD5 checksum and store it on disk upon receipt and storage of the file.  Upon reading of the data file from disk, we want to re-check the MD5 to ensure the data was not corrupted.

Can we use
https://github.com/gcc-mirror/gcc/blob/master/libiberty/md5.c

Or use crypt()/crypt_r()

I saw your question on https://gist.github.com/creationix/4710780, I saw  that code also and was wondering why it didn't have any copyright, etc ...

close issue #60

authors: manderiasian schmidtw

repo: Comcast/aker


text:
Clarification requested regarding Act of Communication, Intentional Process

Suppose I have a camera set up in the forest to monitor flash floods, triggered by a motion sensor and a sound sensor.  When the motion sensor and sound sensor sense activity above a threshold,  a short video is recorded and sent to a server for later analysis. Is that sending of the video an Act of Communication?
The definition of Intentional Action, superclass of Act of Communication is

An Act in which at least one Agent plays a causative role and which is prescribed by some Directive Information Content Entity held by at least one of the Agents.

Some questions that arise:

Is the camera system an agent, i.e. capable of intentional acts?
If so, it would seem that there are many Artifacts that are in fact Agents.
If not, the closest agent to the event are people such as someone who helped design the camera system, someone who helped make it, someone who placed it in the forest, someone who turned it on, someone who ordered it placed. Are any of these realizing a causative role that would satisfy the definition. While these people play a causative role in some sense,  none of their acts are specific to a particular instance of the process described. Nonetheless is it intended that such agents satisfy the Intentional Act criterion "at least one Agent plays a causative role"?
If the answer to (3) is yes then

Are there any communications that are not intentional processes? More generally, are all realizations of Artifact Functions also Acts?
Would all Mechanical Processes be Acts as they involved an Artifact, which involved an Agent that designed the Artifact?
Should the domain of has accomplice, has agent, has accessory, and the subproperties of prescribed by all be Act?
Similarly would any realization of a role that 'has subordinate role' some role be an Act?
Would has accomplice, has accessory, has sender, has recipient all be subproperties of has agent?

There are more questions that arise. Basically, if the answer to (3) is yes then one would look for any mentions of Agent or Act, trace back to a related process, and ask whether that process must be an Act, and review relations whose domain is process and range (explicit or implied by definition) is an agent and ask if they are subproperties of has agent.
I'm assuming that, although unstated, the domain of has agent is Act. Would that be correct?
Depending on some of the answers to these questions, quick fixes could result in asserted polyhierarchy, suggesting some refactoring be done in order to avoid that.

I personally don't think so.
I think the causative role has to played within the process, not upstream of it to be an Act. #79

So I don't think this is an Act of Communication at all.

In #79 I comment on the "causally active" criterion, which I think is difficult to evaluate.
@eliasweatherfield I wasn't clear on which question you your (1) answers. Also, which "this" are you referring to in the last sentence?

Sorry. 1 refers to 1. I don't think the camera system is an agent. "This" refers to the sending of the video.

The case would be more complicated if the camera system was an advanced type of AI having beliefs, desires and intentions.

authors: alanruttenberg eliasweatherfield

repo: CommonCoreOntology/CommonCoreOntologies


text:
🛑 Recap Time Squad Wiki is down

In 942923a, Recap Time Squad Wiki (https://wiki.recaptime.eu.org) was down:

HTTP code: 503
Response time: 298 ms

Resolved: Recap Time Squad Wiki is back up in 92c41cb.

repo: Community-Lores/status-page


text:
Get a Count of Issues Opened/Resolved per user

We should get a count of issues opened/resolved per user!

Fixed by #36

Closed in error

authors: edwardstm gurnben

repo: CompassSoftware/GDET-Tremendous-Trio


text:
Bug: failed to extract payload schema: cannot resolve field type "jsonb"

Bug description
I am trying to start a pipeline that would read rows of my outbox table into Kafka. The actual table contains about 10 columns one of which is jsonb to which I originally attributed the problem I am expiriencing.
However, during my troubleshooting I've created a table that is just a bigint Id and a varchar field and tried reading from it with the same results. I've also dropped the Kafka destination and replaced it with a file one. Even though the destination plugin did not mention any issues.
I am certain that the conenctor reaches the correct table as I originally copied it from the real one but without any constraints. The connector complained about the primary key missing. Adding it to the test table got me back to the original issue.
Conduit is running in Docker, the info endpoint returns:
{   "version": "v0.12.3",   "os": "linux",   "arch": "amd64" }
The connector is of version v0.10.1
Postgres is "PostgreSQL 14.3, compiled by Visual C++ build 1914, 64-bit"
My pipeline file is as follows:
version: 2.2                    # Parser version

pipelines:                      # A list of pipeline configurations
  - id: Obx               # Pipeline ID [required]
    status: running             # Pipeline status at startup (running or stopped)
    name: outbox        # Pipeline name
    description: desc           # Pipeline description
    connectors:                 # A list of connector configurations
      - id: pg
        name: Outbox
        settings:
          url: postgresql://postgres:1@my_ip:5432/db_name?search_path=schema_name
          tables: test_table
         # the following settings were added during my troubleshooting, they do not seem to change the error
          snapshotMode: never
          sdk.schema.extract.payload.enabled: false
          sdk.schema.context.enabled: false
          sdk.schema.extract.key.enabled: false
        type: source
        plugin: builtin:postgres
      - id: con2
        type: destination
        plugin: builtin:file
        name: my-file-destination
        settings:
          path: ./file2.txt
    dead-letter-queue:          # Dead-letter queue (DLQ) configuration
      plugin: "builtin:file"    # DLQ Connector plugin
      settings:                 # A map of configuration keys and values for the plugin (specific to the chosen plugin)
        path: "./dlq.out"
      window-size: 5            # DLQ nack window size
      window-nack-threshold: 2  # DLQ nack window threshold

Here are the full logs as displayed when starting a brand new container:
2024-12-12T13:07:47+00:00 INF All 0 tables opened in 0s component=badger.DB
2024-12-12T13:07:47+00:00 INF Discard stats nextEmptySlot: 0 component=badger.DB
2024-12-12T13:07:47+00:00 INF Set nextTxnTs to 0 component=badger.DB
2024-12-12T13:07:47+00:00 INF loading processor plugins from directory /app/processors ... component=plugin.processor.standalone.Registry
2024-12-12T13:07:47+00:00 WRN could not read processor plugin directory error="open /app/processors: no such file or directory" component=plugin.processor.standalone.Registry
2024-12-12T13:07:47+00:00 INF standalone processor plugins initialized component=plugin.processor.standalone.Registry count=0 plugin_path=/app/processors
2024-12-12T13:07:47+00:00 INF builtin processor plugins initialized component=plugin.processor.builtin.Registry count=17
2024-12-12T13:07:47+00:00 INF processors initialized component=processor.Service count=0
2024-12-12T13:07:47+00:00 INF connector utilities started address=[::]:41251
2024-12-12T13:07:47+00:00 INF connector utilities started on [::]:41251
2024-12-12T13:07:47+00:00 INF builtin connector plugins initialized component=plugin.connector.builtin.Registry count=6
2024-12-12T13:07:47+00:00 WRN could not read connector plugin directory error="open /app/connectors: no such file or directory" component=plugin.connector.standalone.Registry
2024-12-12T13:07:47+00:00 INF standalone connector plugins initialized component=plugin.connector.standalone.Registry count=0 plugin_path=/app/connectors
2024-12-12T13:07:47+00:00 INF connectors initialized component=connector.Service count=0
2024-12-12T13:07:47+00:00 INF pipelines initialized component=pipeline.Service count=0
2024-12-12T13:07:47+00:00 INF pipeline started component=lifecycle.Service pipeline_id=Obx
2024-12-12T13:07:47+00:00 INF pipeline configs provisioned component=provisioning.Service created=["Obx"] deleted=[] pipelines_path=/app/pipelines
2024-12-12T13:07:47+00:00 INF grpc API started address=[::]:8084
2024-12-12T13:07:47+00:00 INF destination connector plugin successfully started component=connector.Destination connector_id=Obx:con2
2024-12-12T13:07:47+00:00 INF http API started address=[::]:8080
2024-12-12T13:07:47+00:00 INF destination connector plugin successfully started component=connector.Destination connector_id=Obx-dlq
2024-12-12T13:07:47+00:00 INF
2024-12-12T13:07:47+00:00 INF click here to navigate to Conduit UI: http://localhost:8080/ui
2024-12-12T13:07:47+00:00 INF click here to navigate to explore the HTTP API: http://localhost:8080/openapi
2024-12-12T13:07:47+00:00 INF
2024-12-12T13:07:47+00:00 WRN Publication "conduitpub" already exists. component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T13:07:47+00:00 WRN replication slot "conduitslot" already exists component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T13:07:47+00:00 INF Starting logical replication at 2/DC3C7E78 component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source publication=conduitpub slot=conduitslot
2024-12-12T13:07:47+00:00 INF Logical replication started component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source publication=conduitpub slot=conduitslot
2024-12-12T13:07:47+00:00 INF source connector plugin successfully started component=connector.Source connector_id=Obx:pg
2024-12-12T13:07:47+00:00 INF source does not support batch reads, falling back to single reads component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T13:07:47+00:00 ERR replication exited with an error error="handler error: logrepl handler insert: failed to update avro schema: failed to extract payload schema: cannot resolve field type \"jsonb\" " component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source stack=null
2024-12-12T13:07:47+00:00 INF source connector plugin successfully torn down component=connector.Source connector_id=Obx:pg
2024-12-12T13:07:47+00:00 ERR node stopped error="node Obx:pg stopped with error: source stream was stopped unexpectedly: error reading from source: read plugin error: failed to fetch next record: logical replication error: handler error: logrepl handler insert: failed to update avro schema: failed to extract payload schema: cannot resolve field type \"jsonb\" " component=lifecycle.Service node_id=Obx:pg stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run","line":146},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run.func1","line":90}]
2024-12-12T13:07:47+00:00 ERR node stopped error="node fanout stopped with error: context canceled" component=lifecycle.Service node_id=fanout stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T13:07:47+00:00 INF destination connector plugin successfully torn down component=connector.Destination connector_id=Obx-dlq
2024-12-12T13:07:47+00:00 INF destination connector plugin successfully torn down component=connector.Destination connector_id=Obx:con2
2024-12-12T13:07:47+00:00 ERR node stopped error="node Obx:con2-metrics stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2-metrics stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T13:07:47+00:00 ERR node stopped error="node fanin stopped with error: context canceled" component=lifecycle.Service node_id=fanin stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T13:07:47+00:00 ERR node stopped error="node Obx:con2-acker stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2-acker stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T13:07:47+00:00 ERR node stopped error="node Obx:con2 stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2 stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]

Steps to reproduce

Have the postgres ready as described above (a table with an id and some varchar column)
docker run -it -p 8080:8080 -v path\to\pipe.yaml:/app/pipelines/pipeline.yaml conduit.docker.scarf.sh/conduitio/conduit
Observe the errors mentioned above

Version
Conduit v0.12.3 connector v0.10.1

@WORMrus Thanks for reaching out to us.:)
So, looking at your logs, I see this:
2024-12-12T13:07:47+00:00 WRN Publication "conduitpub" already exists. component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T13:07:47+00:00 WRN replication slot "conduitslot" already exists component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T13:07:47+00:00 INF Starting logical replication at 2/DC3C7E78 component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source publication=conduitpub slot=conduitslot
...
2024-12-12T13:07:47+00:00 ERR replication exited with an error error="handler error: logrepl handler insert: failed to update avro schema: failed to extract payload schema: cannot resolve field type \"jsonb\" " component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source stack=null

The connector is reading an existing publication slot, and we can also see in the logs the jsonb type mentioned, which means that it's not the table that contains just the big int and a varchar table.
I'd probably do the following:

delete the publication slot (or recreate the Postgres instance)
run Conduit again (from scratch, deleting the old container)

A table with just a primary key, big int and varchar should work, we've been using that often (on the Conduit Platform too). The jsonb column is something that we'd need to check.

@hariso Thank you for returning to me so quickly, indeed, it seems that the current state was related to the original table and its jsonb column. I did see the warnings but was not sure what to make of them as I am not familiar with replication slots\publications as a feature of PG.
As advised, I've deleted the slot with select pg_drop_replication_slot('conduitslot') and a new container was able to start. It however did not capture any changes. It could only create a replica at startup(after I changed the snapshotMode setting back to the default). An additional drop publication conduitpub was needed to make it work for both existing and new data.
With that out of the way, I've once again removed the slot and the publication, deleted the container and added a new jsonb column (not null, '{}'::jsonb as default).
I was able to start a new container. However, it threw an error when I inserted a new row (json there was just a {}):
2024-12-12T22:27:18+00:00 INF All 0 tables opened in 0s component=badger.DB
2024-12-12T22:27:18+00:00 INF Discard stats nextEmptySlot: 0 component=badger.DB
2024-12-12T22:27:18+00:00 INF Set nextTxnTs to 0 component=badger.DB
2024-12-12T22:27:18+00:00 INF loading processor plugins from directory /app/processors ... component=plugin.processor.standalone.Registry
2024-12-12T22:27:18+00:00 WRN could not read processor plugin directory error="open /app/processors: no such file or directory" component=plugin.processor.standalone.Registry
2024-12-12T22:27:18+00:00 INF standalone processor plugins initialized component=plugin.processor.standalone.Registry count=0 plugin_path=/app/processors
2024-12-12T22:27:18+00:00 INF builtin processor plugins initialized component=plugin.processor.builtin.Registry count=17
2024-12-12T22:27:18+00:00 INF processors initialized component=processor.Service count=0
2024-12-12T22:27:18+00:00 INF connector utilities started address=[::]:37225
2024-12-12T22:27:18+00:00 INF connector utilities started on [::]:37225
2024-12-12T22:27:18+00:00 INF builtin connector plugins initialized component=plugin.connector.builtin.Registry count=6
2024-12-12T22:27:18+00:00 WRN could not read connector plugin directory error="open /app/connectors: no such file or directory" component=plugin.connector.standalone.Registry
2024-12-12T22:27:18+00:00 INF standalone connector plugins initialized component=plugin.connector.standalone.Registry count=0 plugin_path=/app/connectors
2024-12-12T22:27:18+00:00 INF connectors initialized component=connector.Service count=0
2024-12-12T22:27:18+00:00 INF pipelines initialized component=pipeline.Service count=0
2024-12-12T22:27:18+00:00 INF pipeline started component=lifecycle.Service pipeline_id=Obx
2024-12-12T22:27:18+00:00 INF pipeline configs provisioned component=provisioning.Service created=["Obx"] deleted=[] pipelines_path=/app/pipelines
2024-12-12T22:27:18+00:00 INF grpc API started address=[::]:8084
2024-12-12T22:27:18+00:00 INF destination connector plugin successfully started component=connector.Destination connector_id=Obx:con2
2024-12-12T22:27:18+00:00 INF destination connector plugin successfully started component=connector.Destination connector_id=Obx-dlq
2024-12-12T22:27:18+00:00 INF http API started address=[::]:8080
2024-12-12T22:27:18+00:00 INF
2024-12-12T22:27:18+00:00 INF click here to navigate to Conduit UI: http://localhost:8080/ui
2024-12-12T22:27:18+00:00 INF click here to navigate to explore the HTTP API: http://localhost:8080/openapi
2024-12-12T22:27:18+00:00 INF
2024-12-12T22:27:18+00:00 INF Starting logical replication at 2/DC882DC8 component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source publication=conduitpub slot=conduitslot
2024-12-12T22:27:18+00:00 INF Logical replication started component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source publication=conduitpub slot=conduitslot
2024-12-12T22:27:18+00:00 INF source connector plugin successfully started component=connector.Source connector_id=Obx:pg
2024-12-12T22:27:18+00:00 INF source does not support batch reads, falling back to single reads component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source
2024-12-12T22:27:37+00:00 ERR replication exited with an error error="handler error: logrepl handler insert: failed to update avro schema: failed to extract payload schema: cannot resolve field type \"jsonb\" " component=plugin connector_id=Obx:pg plugin_name=builtin:postgres plugin_type=source stack=null
2024-12-12T22:27:37+00:00 INF source connector plugin successfully torn down component=connector.Source connector_id=Obx:pg
2024-12-12T22:27:37+00:00 ERR node stopped error="node Obx:pg stopped with error: source stream was stopped unexpectedly: error reading from source: read plugin error: failed to fetch next record: logical replication error: handler error: logrepl handler insert: failed to update avro schema: failed to extract payload schema: cannot resolve field type \"jsonb\" " component=lifecycle.Service node_id=Obx:pg stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run","line":146},{"file":"/app/pkg/lifecycle/stream/source.go","func":"github.com/conduitio/conduit/pkg/lifecycle/stream.(*SourceNode).Run.func1","line":90}]
2024-12-12T22:27:37+00:00 ERR node stopped error="node fanout stopped with error: context canceled" component=lifecycle.Service node_id=fanout stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 ERR node stopped error="node Obx:con2-acker stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2-acker stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 INF destination connector plugin successfully torn down component=connector.Destination connector_id=Obx-dlq
2024-12-12T22:27:37+00:00 ERR node stopped error="node Obx:con2-metrics stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2-metrics stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 INF destination connector plugin successfully torn down component=connector.Destination connector_id=Obx:con2
2024-12-12T22:27:37+00:00 ERR node stopped error="node Obx:pg-metrics stopped with error: context canceled" component=lifecycle.Service node_id=Obx:pg-metrics stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 ERR node stopped error="node fanin stopped with error: context canceled" component=lifecycle.Service node_id=fanin stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 ERR node stopped error="node Obx:con2 stopped with error: context canceled" component=lifecycle.Service node_id=Obx:con2 stack=[{"file":"/app/pkg/lifecycle/service.go","func":"github.com/conduitio/conduit/pkg/lifecycle.(*Service).runPipeline.func2","line":743}]
2024-12-12T22:27:37+00:00 INF restarting with backoff attempt=1 component=lifecycle.Service duration=1917.753816 pipeline_id=Obx

So it seems that something is off with jsonb columns

authors: WORMrus hariso

repo: ConduitIO/conduit-connector-postgres


text:
eslint config?

@skmgoldin I see you added some eslint plugins. Do you have a config file for it?
Should we add eslint as a dependency as well, or is best practice to run a global install?

Good catch we should install eslint itself in the devDependencies. I'll look into this.

Looks like it may have been eslint's standard rules: https://github.com/ConsenSys/Tokens/blame/59e92af4a290e0d7945084e78daa4a57cecffe9b/package.json#L36

authors: maurelian skmgoldin

repo: ConsenSys/Tokens


text:
Version API docs

The Tessera API is versioned (see https://github.com/ConsenSys/tessera/pull/1206) but it appears only 'master(latest' can be selected as the version on the API reference (https://consensys.github.io/tessera/)

Writing a Github Action in progress...

Writing a Github Action in progress...

authors: MadelineMurray NicolasMassart


text:
Feat: unified ECADD

Fixes #630.
→ Suggested solution:
~~Select λ to be either (y1-p.y2)/(x1-x2) or 3x1²/2y1 based on x1==x1 && y1==y2. The problem is that division by x1-x2 will trigger an error as it uses big.Int ModInverse https://github.com/ConsenSys/gnark/blob/c02da5f61944d13c557d2b390ca56d5a26c597a1/std/math/emulated/hints.go#L251~~
So we implement a special method DivSpecial() that returns 0 when denominator is 0. We use it to compute (y1-p.y2)/(x1-x2) and proceed further, and then select 3x1²/2y1 if applicable.
Actually we can use Brier-Joye unified addition formulas, which compute a slope that works for both addition and doubling in affine coordinates — and also save half the constraints of the previous solution.

We use zero=(0,0) as the affine "representation" of the infinity point.

[x] UnifiedAdd(P, zero) should output P
[x] UnifiedAdd(zero, P) should output P
[x] UnifiedAdd(zero, zero) should output zero
[x] UnifiedAdd(P, -P) should output zero
[x] ScalarMul(0, P) should output zero
[x] ScalarMul(s, zero) should output zero
[x] ScalarMulBase(0) should output zero

These methods are used in the EVM precompiles, so they should follow the (0,0) convention.
For double(), triple(), add() and doubleAndAdd() (now private) we don't handle the edge cases (used when these should not happen).
For Pair() the zkEVM arithmetization would filter the (0,0) points @OlivierBBB.

fixes #634

Suggested edit:
diff --git a/std/algebra/emulated/sw_emulated/point.go b/std/algebra/emulated/sw_emulated/point.go
index 69e32594..0ef6cd01 100644
--- a/std/algebra/emulated/sw_emulated/point.go
+++ b/std/algebra/emulated/sw_emulated/point.go
@@ -126,12 +126,12 @@ func (c *Curve[B, S]) add(p, q *AffinePoint[B]) *AffinePoint[B] {
 //
 // ✅ p can be equal to q, and either or both can be (0,0).
 // (0,0) is not on the curve but we conventionally take it as the
-// neutral/infinity point as per the EVM [EYP].
+// neutral/infinity point as per the [EVM].
 //
-// It uses the unified formulas of Brier and Joye [BriJoy02] (Corollary 1).
+// It uses the unified formulas of Brier and Joye ([[BriJoy02]] (Corollary 1)).
 //
 // [BriJoy02]: https://link.springer.com/content/pdf/10.1007/3-540-45664-3_24.pdf
-// [EYP]: https://ethereum.github.io/yellowpaper/paper.pdf
+// [EVM]: https://ethereum.github.io/yellowpaper/paper.pdf
 func (c *Curve[B, S]) AddUnified(p, q *AffinePoint[B]) *AffinePoint[B] {

 	// selector1 = 1 when p is (0,0) and 0 otherwise
@@ -338,7 +338,7 @@ func (c *Curve[B, S]) Lookup2(b0, b1 frontend.Variable, i0, i1, i2, i3 *AffinePo
 //
 // ✅ p can can be (0,0) and s can be 0.
 // (0,0) is not on the curve but we conventionally take it as the
-// neutral/infinity point as per the EVM [EYP].
+// neutral/infinity point as per the [EVM].
 //
 // It computes the standard little-endian variable-base double-and-add algorithm
 // [HMV04] (Algorithm 3.26).
@@ -352,7 +352,7 @@ func (c *Curve[B, S]) Lookup2(b0, b1 frontend.Variable, i0, i1, i2, i3 *AffinePo
 //
 // [ELM03]: https://arxiv.org/pdf/math/0208038.pdf
 // [HMV04]: https://link.springer.com/book/10.1007/b97644
-// [EYP]: https://ethereum.github.io/yellowpaper/paper.pdf
+// [EVM]: https://ethereum.github.io/yellowpaper/paper.pdf
 func (c *Curve[B, S]) ScalarMul(p *AffinePoint[B], s *emulated.Element[S]) *AffinePoint[B] {

 	// if p=(0,0) we assign a dummy (0,1) to p and continue
@@ -402,7 +402,7 @@ func (c *Curve[B, S]) ScalarMul(p *AffinePoint[B], s *emulated.Element[S]) *Affi
 //
 // ✅ When s=0, it retruns (0,0).
 // (0,0) is not on the curve but we conventionally take it as the
-// neutral/infinity point as per the EVM [EYP].
+// neutral/infinity point as per the [EVM].
 //
 // It computes the standard little-endian fixed-base double-and-add algorithm
 // [HMV04] (Algorithm 3.26).
@@ -413,7 +413,7 @@ func (c *Curve[B, S]) ScalarMul(p *AffinePoint[B], s *emulated.Element[S]) *Affi
 // [3]g, [5]g and [7]g points.
 //
 // [HMV04]: https://link.springer.com/book/10.1007/b97644
-// [EYP]: https://ethereum.github.io/yellowpaper/paper.pdf
+// [EVM]: https://ethereum.github.io/yellowpaper/paper.pdf
 func (c *Curve[B, S]) ScalarMulBase(s *emulated.Element[S]) *AffinePoint[B] {
 	g := c.Generator()
 	gm := c.GeneratorMultiples()

Suggested edit:
diff --git a/std/algebra/emulated/sw_emulated/point.go b/std/algebra/emulated/sw_emulated/point.go
index 69e32594..78b05b99 100644
--- a/std/algebra/emulated/sw_emulated/point.go
+++ b/std/algebra/emulated/sw_emulated/point.go
@@ -400,7 +400,7 @@ func (c *Curve[B, S]) ScalarMul(p *AffinePoint[B], s *emulated.Element[S]) *Affi
 // ScalarMulBase computes s * g and returns it, where g is the fixed generator.
 // It doesn't modify s.
 //
-// ✅ When s=0, it retruns (0,0).
+// ✅ When s=0, it returns (0,0).
 // (0,0) is not on the curve but we conventionally take it as the
 // neutral/infinity point as per the EVM [EYP].
 //

Applied the changes. If looks good then go for merge.

authors: ivokub yelhousni

repo: ConsenSys/gnark


text:
Change timeout logic

Changes introduced:

Increase default execution timeout to 24h (i.e. usually fully execute two tx-count transactions)
When execution-timeout is set, instead of dividing time between transactions, attempt to complete execution of each tx until the timeout.

This shouldn't cause any difference in tyro right? Also a print message to stdout saying analysis is in progress(and warn them about the execution timeout) might be some help, as some users raised issues saying that mythril froze, if the default is switched to 24hrs then the users would definitely assume thats the case.

We'd need to adapt tyro configurations

This shouldn't cause any difference in tyro right?

The API behavior shouldn't change because we always provide timeouts. @JoranHonig I don't think we need to change anything in Tyro?

as some users raised issues saying that mythril froze

If users are concerned about analysis progress they can add the -v4 flag. Analysis is expected to take long for anything >-t1.

@b-mueller timeouts would behave differently right? each transaction would be allowed to run for the entire execution-timeout, whereas now the execution-timeout sets a bound on the global time taken for the analysis.
So the analysis can take N times as long, where N is the amount of transactions we tell mythril to do

I'm not sure how this got in there but there seem to be both self.time_handler and self.time being used for timout purposes. The time handler probably also needs to be changed to allow this pr to work.
However I think we should go with either using the time handler or the time variable, not both

the execution-timeout sets a bound on the global time taken for the analysis.

That's how it works in this PR, the timeout applies to the entire analysis. At least that's what I intended.

I'm not sure how this got in there but there seem to be both self.time_handler and self.time being used for timout purposes. The time handler probably also needs to be changed to allow this pr to work

On a quick glance it seems that time handler doesn't need changing, even though I'm not sure if the way it's used makes sense. Currently we use it to dynamically set the solver timeout to time_remaining() - 500. I'd rather not have the solver use 52 minutes of a 1 hour analysis if it gets stuck on a difficult task.
Why not just use a fixed timeout?

@b-mueller , It's mostly max(100s, time_remaining()-500) for solving, that was done for the sake of making execution-timeout flag proper. We couldn't kill mythril when it is solving, as it results in some segfaults, so had to use this way.

Is there anything that prevents merging this? Also, does it conflict with #958?

authors: JoranHonig b-mueller norhh

repo: ConsenSys/mythril-classic


text:
Mythril docker container getting closed with code 0

When i run a docker container it gets stopped without returning output in few seconds giving "exited with code 0"

Yes. But shouldn't there be a command,
something like docker run mythril/myth version

repo: ConsenSys/mythril


text:
solution for periods in subgraph name breaking dot

Problem: If a subgraph name was generated for a variable it would break the bash piping to dot.
Solution: wrapped the subgraph name in double quotes so that
subgraph clusterContractName.VariableName
becomes
subgraph "clusterContractName.VariableName"

Thank you @speakerwiggin ! 😄 🙇  You rock!

authors: GNSPS speakerwiggin

repo: ConsenSys/surya


text:
[Merge] unhide --Xvalidators-proposer-*

PR Description
unhide --Xvalidators-proposer-*
Fixed Issue(s)
#4874
Documentation

[ ] I thought about documentation and added the doc-change-required label to this PR if updates are required.

Changelog

[ ] I thought about adding a changelog entry, and added one if I deemed necessary.

One more to unhide please:

oh gosh is on the top 😭

repo: ConsenSys/teku


text:
Bonsai Tree Verifier improvements

MVP improvements to enable us to verify canary nodes.
What this PR does:

Upgrades project to java 21 and gradle 8
Adds bonsaitreeverifier binary to bela/bin/ distro alongside bela binary (so it can be run via CLI)
Code by code hash support (falls back to code by account hash)
Better surfacing of errors upon traversal completion
Display duration at end of traversal (about 8 hours for mainnet)

What this PR does not do (but I'd like to do in future PRs):

Does not fix compatibility with recent versions of besu (I've been using ./gradlew installDist -PbesuVersion=23.7.2 to build and reverting the database metadata file)
Does not improve traversal performance
Does not fix issues running Bonsai Tree Verifier via the lanterna UI
Does not enable read only mode for verifier

Running this code a canary:
See https://github.com/Consensys/protocol-misc/issues/972#issuecomment-2325590845
cd $HOME
git clone https://github.com/Consensys/bela
cd bela
git pull
git checkout verifier-improvements
./gradlew installDist -PbesuVersion=23.7.2
sudo /opt/besu/current/bin/besu --data-path=/data/besu storage revert-metadata v2-to-v1
sudo service besu stop
nohup sudo env USER_HOME=$HOME bash -c '{ time $USER_HOME/bela/build/install/bela/bin/bonsaitreeverifier /data/besu > /data/bela.log 2>&1; }' &

Example of unsuccessful run:
We are verifying : /tmp/besu

ޏ₍ ὸ.ό₎ރ
🤞🤞🤞🤞🤞🤞🤞
Starting from root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544
Working with root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544

missing account trie node for hash 0xd1aca5cee11650c81f5fafb518f320105c4e7ad931a3e46b7ee73b05476bd3d3 and location 0x03020808
missing account trie node for hash 0xf33ad4d20fc5fae3dcc1a19afc5a510ebc10a37c7c3e0d89bddef33bd698f454 and location 0x0d0f0903
missing account trie node for hash 0x84269463e5e9ecf08491d8745b98cec308498076c2cacbbe1c6e7adbe5d00438 and location 0x0f0f0e

Found 3 errors (see above)

Could not verify root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544 with 3462 nodes
AAAAAAAAAA!!!!!!!
Duration: 0 hours, 0 minutes, 1 seconds

Example of successful run:
We are verifying : /tmp/besu

ޏ₍ ὸ.ό₎ރ
🤞🤞🤞🤞🤞🤞🤞
Starting from root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544
Working with root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544

ޏ₍ ὸ.ό₎ރ World state was verified... ޏ₍ ὸ.ό₎ރ
Verified root 0xd7f8974fb5ac78d9ac099b9ad5018bedc2ce0a72dad1827a1709da30580f0544 with 3463 nodes
AAAAAAAAAA!!!!!!!
Duration: 0 hours, 0 minutes, 2 seconds

🚢   !

authors: garyschulte siladu

repo: Consensys/bela


text:
[FRID-206] As an inventory engineer, so I can scan newer repos, update LicenseFinder to the latest version from Pivotal and get it working.

As an inventory engineer, so I can scan newer repos, update LicenseFinder to the latest version from Pivotal and get it working.
https://grandrounds.atlassian.net/browse/FRID-206
Mostly the merge from Pivotal, but it took a few commits to get it to scan again.

I'm merging without reviewers because (a) nobody but me has ever touched this code, (b) the reviews have been standing waiting for more than six months, and most importantly (c) this isn't production code and will not be deployed to any production system, so can be merged without review.

repo: ConsultingMD/LicenseFinder


text:
API for adding additonal players to ReactPlayer in runtime

#362

Nice one! Thanks @BrooklynKing and @AndyBarron.

Published in 1.4.0. Sorry for the delay!

@BrooklynKing @AndyBarron could either of you share an example of successfully using a custom player with this API?

@shawninder not sure i can share example, since it's in private repo, but i can try to answer on your questions

I'm getting weird errors I can't figure out with my custom player, yet it's basically copy-pasted from the FilePlayer... Here's a question. Did you also use createSinglePlayer as react-player uses for its players, or did you start from scratch?
If you're curious, here is my work-in-progress: https://github.com/shawninder/music/blob/master/components/IndexedDBPlayer.js It currently doesn't do anything (I was hoping to stop the Component from crashing the page before starting to add functionality) as it's just the FilePlayer copied, renamed, and with the File stuff removed. I just don't understand why this crashes the page whereas the FilePlayer included in react-player doesn't.

@shawninder are you doing
import { IndexedDBPlayer } from '...'

or
import IndexedDBPlayer from '...'

as the latter would cause issues.

Ah, damn it, I am using import IndexedDBPlayer from '...'. Thanks @CookPete, I'm sure this is my problem (will confirm tomorrow). It's times like these when I miss working with a team.

@CookPete While you're in the area, here's a question for you: If I do get this IndexedDBPlayer working, would you be interested in a pull request adding the player to this repo? I'm guessing not, since playing media from IndexedDB feels like an infrequent use-case, but if you do want it, just tell me and I'll submit a pull request when I'm "done".

authors: BrooklynKing CookPete shawninder

repo: CookPete/react-player


text:
⚠ Samsung has degraded performance

In a8129f9, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2843 ms

Resolved: Samsung performance has improved in 8b630b2.

repo: CoopPlayzz/Webstatus-electronicbrands


text:
⚠ Samsung has degraded performance

In 117e6be, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2284 ms

Resolved: Samsung performance has improved in 5529e21.


text:
⚠ Samsung has degraded performance

In 4d68da0, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2940 ms

Resolved: Samsung performance has improved in 3de6553.


text:
⚠ Samsung has degraded performance

In c64a1e1, Samsung (https://samsung.com) experienced degraded performance:

HTTP code: 200
Response time: 2490 ms

Resolved: Samsung performance has improved in f85b8c1.


text:
⚠ Fortnite has degraded performance

In ebbcdc0, Fortnite (https://fortnite.com) experienced degraded performance:

HTTP code: 200
Response time: 1577 ms

Resolved: Fortnite performance has improved in 62f4f7d.

repo: CoopPlayzz/Webstatus-games


text:
⚠ ESPN has degraded performance

In d88f195, ESPN (https://espn.com) experienced degraded performance:

HTTP code: 200
Response time: 2277 ms

Resolved: ESPN performance has improved in fb72ff0.

repo: CoopPlayzz/Webstatus-sports


text:
Can we assume one DBL?

In today's call, we were operating under the assumption that there is no central naming authority.  Could DBL serve as a central naming authority?  I have been assuming that we plan to have one DBL, not many, and that assumption is found in some parts of our documentation:
https://docs.burrito.bible/en/latest/history.html#new-media-new-ecosystems

While Paratext will continue to upload much of the text content of DBL, uploading of other entry types is now handled by Nathanael, an application developed by the DBL team. Bible text editors such as Autographa and translationStudio are being developed to meet the needs of teams with modest language and technology skills. Groups such as unfoldingWord, Wycliffe Associates and Faith Comes By Hearing are developing ecosystems for users of their own systems. This is the context within which the ETEN Bible Tools committee identified a need for data exchange formats, leading to Scripture Burrito.

But in other places our documentation does not seem to assume a single DBL:

Scripture Burrito assumes the existence of ecosystem servers that provide ids for users, organizations and projects, and stores information to enable that server-hosted context to be discovered.
Scripture Burrito is intended to allow lossless roundtripping of projects between ecosystems. This depends to some extent on references to ecosystem servers that enable reconnection with different ecosystem-specific contexts.

That description seems to imply that there might be multiple co-equal DBLs.   If that's the case, we have other problems to solve if we want to have a good overview of what is available and how it is licensed.
Which is ETEN asking us to do?  This needs to be clear in our documentation.  It may inform how we approach things like identifiers.

To me, this is really a question about silos versus building a community.
DBL is designed to include Scripture from many sources, providing an overview that lets the translation community and donors see the current status and which needs are most acute, shows people who want to use Scripture who holds the copyright so they can negotiate, etc.
If DBL is just another silo, we would need to build something else for federated searches across silos.   But the whole purpose of DBL is to be that shared space.
Do we agree on that?  If not, is there any notion of a shared space that we are trying to build together?

Also from the docs,

Scripture Burrito is designed first and foremost for data interchange between ecosystems...

Nothing in SB is meant as a replacement of DBL, nor is anything in SB intended to take away from DBL. On the contrary, it's intended to make it much simpler for developers to get data into and out of DBL.
The conversation is important in terms of discoverability of projects, which has come up from time to time. Once we have SB defined and in place, it becomes straightforward to get projects in and out of systems and a conversation around discoverability is likely to get much more traction. In other words, once we have a common data format, it becomes easy to standardize servers/apis on that.
Another note is that most projects start in other ecosystems because DBL is focused on archival and distribution of Bible translations. Further, the SB scope of resources is much larger than the (current) DBL scope of only Scripture. I can't speak for their long terms goals but I don't think anything in the peripheral flavor would be added to the DBL.

Here's one way to position this:

For Scripture (source language texts and translations), SB allows applications in the DBL ecosystem to exchange data with DBL and with each other.  DBL makes these resources discoverable.  We should think about how the lifecycle of identifiers relates to this.

For things that do not look like Scripture, we have the same discoverability needs that DBL addresses for Scripture.  The Copenhagen Alliance needs to look at interoperability and discoverability for these resources.  Since all of these resources are related to Scripture, should we discuss whether we need a place in DBL for these as well?  Or something else like DBL?

@jonathanrobie On that specific proposal, it seems to me that, on the one hand, we lose the benefits of having everything linked to a single place and, on the other hand, we don't get the flexibility that comes with supporting an arbitrary number of reference servers.
The scope of SB ended up being very large indeed, so I think it would be a stretch to imagine that anything anyone could ever represent as a SB would make sense within DBL. There's also the issue that DBL has policies that might prevent some organizations from joining (eg if there are unresolved conflicts with one of the major stakeholders.) I think that's entirely reasonable. What wouldn't be reasonable is for DBL to be able to effectively stop such organizations from translating Scripture, which might be the unintended corollary of making DBL the sole reference for everything.
Personally, I think that software design should be apolitical. I think there are some issues that do need to be worked through, maybe by people above my pay grade, but I don't think we need to try to anticipate such policies in how we design SB.

There is a set of things that DBL is prepared to handle and that DBL consumers can understand because they are in well-defined formats.  The benefit of seeing beyond individual silos is very important, and that's what DBL does for these resources.
We don't (yet) have an equivalent for other formats.  There is no interoperability story for an arbitrary proprietary format, and we have no discoverability story for things that are not in DBL.  These are things that the Copenhagen Alliance might look at, we may try to persuade DBL to accept more kinds of data, we may want to come up with standards to allow a wider variety of data to be used interoperably (again, something the Copenhagen Alliance might be involved in defining).   But I don't think merely accepting any proprietary format is enough for either interoperability or discoverability.
I agree with you that software design should be apolitical.  Or more precisely: we are designing software to encourage collaboration and sharing, two very political activities.  Clearly, we dont' want to stop anyone from translating Scripture, but I don't think there is any danger of that.  If there is, let's flesh out the use cases and requirements that we need to satisfy to avoid doing that.

I think that, for a lot of possible future SB flavors, it will be very hard to persuade DBL to be involved. That's not about how proprietary the format is, it's about how much like Scripture the content is. It's the Digital Bible Library, not a Digital Christian Library. I think the Bible translation world needs collaborative options that don't depend on policy changes at DBL.
I also think that we need "more than one DBL" just to handle the existence of Paratext servers. There are certainly things that DBL and PT should be doing the same way, but there are also things that should remain different, often because DBL and PT are publication and translation-facing respectively. Designing things to assume that DBL is the centre for everything seems awkward to me even when just dealing with Paratext. For example, there may be translations in Send and Receive that should not be circulated any wider than necessary. Two servers doubles the risk of exposure (potentially with life-changing consequences for the translators).

@mvahowe On your second paragraph -- I'm not sure I understand something. It seems like you're referring to other servers like Send / Receive as other "DBL"s (?). I don't think anyone is assuming or suggesting that a translation project in process (on Send / Receive for example) is automatically (also) stored in DBL. It could be, as you know, and often is, but normally as a step taken when the project owner identifies that some / all of the project content is 'publishable'.
I'm sorry if I'm missing something - I know I have not been engaged in the dialogues happening here.

@mvahowe I don't understand what you mean when you say we need more than one DBL to handle the existence of Paratext servers.   I don't think that Paratext servers do the same thing as DBL.
For Scripture, at least, I think there is only one DBL.  It gives an overview of the status of translation everywhere, across our silos, whether or not people use Paratext.  It also tells you who owns the license and whether it is freely licensed (getting this last part right really matters, there is more to do here).    I don't think it makes sense to have multiple parallel DBLs, if we do that then we have created a set of silos and we need to invent another DBL to help us see what is in the various silos.  The problem turtles if someone chooses not to be part of the new meta-DBL.
The Paratext servers are not DBL, they have a different purpose - providing a platform for collaboration and sharing.   That includes freely sharing many resources with translators even if they are not freely licensed for everyone.   These servers can be used by other clients, but other translation clients can choose to store their data in other places, such as GitHub, and may not be interested in content that is not freely licensed.   This is one place that we really need to allow for multiple ecosystems at the same level.
But we also need to provide for things that do not fit in DBL.  That's important.  The Copenhagen Alliance needs this, and it needs interoperability for these resources.   We also need to discuss discoverability.  Saying that something belongs to the peripheral flavor doesn't really address these requirements.
I would really like to see use cases for all of these things, and careful consideration of how these various systems work with various roles in the user community.   (I won't be at the meeting this week, sorry, I am on vacation.)

@jonathanrobie @klassenjm "One DBL" was not my choice of vocabulary, and I think it's ambiguous. The distinction that I think we need to make is something like the difference between git (an open source program that anyone can use however they like, and which provides consistent interfaces) and GitHub (one particular instance of git, plus some closed-source code that, together, drives the biggest git-powered community.)
There is clearly "only one DBL" if only because I'm sure UBS and/or ETEN think they own that name. Right now, DBL has a unique software stack, but I don't think that's what makes it unique. (There's really nothing that DBL does that is a huge technical challenge, it's all relatively standard CRUD server stuff.)
What makes DBL unique is surely the community of organizations who have chosen to place their content in DBL. DBL without those organizations would be one more Bible enthusiast website. It's not the technology that makes DBL unique. (I often think that DBL occupies that position in spite of some of the technology.)
I'm not suggesting that anyone should be working to set up a dozen DBL competitors. (That could happen, I don't think that obfuscating Scripture Burrito would slow down anyone who wanted to do that, and I think that the formidable challenges are on the level of organizational buy in rather than in writing code.)
What I've been saying from the very first document I wrote for this group is that portability requires formats, but also interfaces. Right now DBL has a unique API. That's not a feature, it's a total pain for anyone who wants software to work with both DBL plus anything else. Consistent interfaces are A Good Thing. That's why we have TWAIN for scanners and PCL and Postscript for printers. That's why we all use a convoluted protocol called TCP/IP to connect any two computers for any reason, rather than point to point connections using some bespoke protocol like we did in the 80s. (We use TCP/IP even though point to point had some undeniable technical advantages such as security and QoS. That's how much interfaces matter.)
If we strip DBL back to the bare technology it is (among other things) a way to store, upload and download Scripture-related content. Paratext has one of those too, but everything about it works a different way. Some of those differences are for good reasons. Many of those differences are accidental, historical or just plain perverse. As a result, it's incredibly hard for anyone to write a third-party application or website that exchanges content with both DBL and the Paratext ecosystem, because much of the code has to be written twice, and then maintained. I'm telling you that there's no way to spin that as a feature, unless the Tower of Babel was a really neat idea to improve international cooperation.
So yes, there's one "thing" called DBL, and I don't expect that to change. But DBL is also one instance of a class of servers that do similar things for the Bible translation and publishing world. As far as I'm concerned, servers within that class should do similar things in similar ways. That saves time and money writing the code in the first place, and it encourages higher adoption of all servers, including DBL, as well as allowing reuse of clients such as as Nathanael.
In that sense, Send and Receive is "another DBL". Or, if you prefer, DBL is "another Send and Receive". And Unfolding Word, Faith Comes by Hearing and other organisations already have products in that space. In that sense, DBL and Send and Receive are "another Faith Comes by Hearing". All I'm saying is that, on a technical level, it would be nice if the servers in that space offered a similar interface for similar operations. That doesn't mean every website becomes DBL. It just means that the next generation of techies will waste a lot less time, that new initiatives will happen faster and that more data will be shared.

Some real issues (mainly overheard at the BT conference in Dallas earlier this year) that the above scenario addresses...

API.Bible's digital publishing tool could work from DBL, as at present, for published work, but could also work from PTX, for work in progress that DBL is not ready to receive. Right now, the options are to force unfinished work into DBL or for API.Bible to run two parallel toolchains.

Publishing Assistant could use texts from DBL, which makes a lot of sense if the text has been licensed. I'm not going to try to describe what happens now, let alone some of the workarounds I heard, all because DBL and Send and Receive are incompatible on every level.

Scripture App Builder could work with DBL texts.

Another instance of YouVersion could work with unpublished texts.

Glisten could use text from either DBL or PTX. (They used to do one, now they do the other, and the choices seem to be on less than solid technical grounds.) They could get audio files from DBL or FCBH or Biblica and mash them up with text from DBL or PTX or Unfolding Word, all with the same codebase.

Collecting aggregate data about both translations and publications, eg for progress.bible, would be way easier.

And so on.

And, to get back to what I think was the specific conversation that sparked this conversation, I think it would be a big mistake to design Scripture Burrito with the assumption that DBL will provide all the ids for every Scripture Burrito document in the world because

I don't think DBL is looking to take on that amount of work
It makes working without an Internet connection harder
It makes intranet and test scenarios harder (ie are we going to have dev builds of "The One DBL" so that third-party developers have a way to test their code without connecting to DBL? Wouldn't that be completely surreal?)
It either means that DBL has to say "yes" to every request for an id or, alternatively, DBL becomes the gatekeeper for any technology using Scripture Burrito, with the power to lock translations or organizations out of any SB technology just by refusing to grant an id.
Either way, DBL then needs some sort of id conflict resolution procedure...

I liked the idea of a delegated URN scheme as discussed previously, but that idea was rejected last week. The only sane alternative I see is supporting arbitrary id servers, and assuming/hoping that common sense and/or Christian virtue will prevent 1000 of these from competing for world domination.
If we wanted a centralized id scheme (and there are certainly technical arguments for that), I don't think it's helpful for tha centralized id scheme to be called DBL.

I don't think that DBL should create all IDs for all systems.  I do think DBL could maintain a registry of prefixes that are recognized for names so that we do not have multiple parties using the same prefix. A delegated ID scheme is good, but a prefix registry makes it better.
I also think it would be good for our documentation to be clear that SB is being designed for systems to exchange data with DBL and with each other and to provide use cases.  And maybe we should have lists of use cases - is there a place to submit them?

I think that a prefix registry is a good idea, which we have discussed before. I don't think that DBL should be the place where that happens. In a sense, the place where that happens right now is the Scripture Burrito schema which has a whitelist of prefixes. If we want a prefix registry and a process to declare prefixes, that sounds to me like a new server that ought to be independent of DBL and PTX and everyone else.
Also, if that's what we want, why wouldn't we make the curated prefix system a URN?

I don't think we can update the Scripture Burrito schema whenever someone registers a new prefix, and that would require the SB group to approve the change.
What is the advantage of having a new server that is independent of DBL for this purpose? If prefix clashes occur, they will occur in DBL.  If people are not putting their texts into DBL, clashes will not be seen.
For non-Scripture resources, we may need an additional solution.  But I really hate to standardize that solution before we work through the use cases and requirements.  And I think that has to include the broader Copenhagen Alliance community. Right now, that part of the spec looks like hand waving to me.

A lot depends how fast prefixes move. I'm expecting initial updates to the Scripture Burrito schema to happen quite regularly for a while, and I'd expect to have most of the organizations we know covered before 1.0. We also have an x- feature which means that organizations can literally make stuff up while waiting for an update.
We can, of course, remove the whitelist from the schema, but then we can't use the schema to validate the prefix, or indeed the org-dependent part of the id. (Right now the DBL schema does check that, eg, a Biblica ID looks like a Biblica ID.)
Prefix clashes can occur anywhere. They could happen in PT Reg. You seem to be assuming that DBL acts as a hub, but it really doesn't. There are probably hundreds or thousands of entries in Send and Receive that have not made it to DBL. I can't imagine Autographa and BTI ever sharing word alignment information via DBL. Maybe the problem is the name DBL, which suggests that it's The (one true) Digital Bible Library. That's not the current reality and I can't see it ever becoming the reality. In Bible lifespan terms, PTReg is much closer to the centre than DBL.
The advantage of an independent server is that it's independent (or, more realistically, that there's an intentional form of governance that is acceptable to all parties.) This is really an issue for @klassenjm  but I can't see UBS wanting to end up having to arbitrate between, say, SIL and Unfolding Word on prefixes or anything else. One of the benefits of the URN route was that it would have imposed a "seen to be fair" process. Not taking the URN route doesn't let us off the hook - we still need to tick the "seen to be fair" box.
Also, on a technical going-on trend level, micro-services are currently preferred over "do it all" monoliths. An independent prefix service would do one quite simple thing, and it ought to be able to do it in a highly scalable and redundant way, with no possibility of, eg that service taking the entire Bible ecosystem down because a Digital Academy workshop is DDoSing DBL with uploads.

(Which bit of the spec is "hand waving"? Last time I looked it spelled out very clearly how ids are constructed, and all the options are currently in the schema. That seems quite specific to me.)

The advantage of an independent server is that it's independent (or, more realistically, that there's an intentional form of governance that is acceptable to all parties.)

This is precisely my point.  DBL has a system of governance.  If we set up an independent server for this purpose, we need to create a system of governance for it.  You can't get around the need for a form of governance.
If we want to work together well, we need systems that go beyond our individual silos.  Putting something in GitHub does not make it discoverable in an organized way or help people get an overview of what is out there.  Any system we create for this purpose will require a system of governance that is acceptable to the people who use it.  There will always be someone who chooses not to participate or needs to be excluded for some important reason.  We want to include as many people as we can.
So if we create a new server with a new system of governance, why would we expect its governance body to be better than ETEN or the Copenhagen Alliance?  Why not use existing organizations for this?

I can't comment on Copenhagen Alliance. I'm pretty sure ETEN doesn't want to directly run anything on this level. DBL governance is pretty much UBS, and that seems to fine for me as long as everything is going fairly well. It's obviously not fine at all if there's a conflict between UBS and another organization. I think PTX has more independent governance than DBL.

PT governance is the Paratext Alliance, which is UBS and SIL, but which actively tries to include and process input from all Paratext users and using organizations. This is via the Paratext BoG in which there are members of SIL and UBS. The DBL governance is less clear -- but primarily in terms of decision making around application process (to contribute or request content) it has been more broad. It is not just UBS. It has been through the PCC (Partnership Credentials Committee), which is composed of representatives from UBS, SIL, Seed Co, Biblica, ETEN, (at least, and maybe another).
At this moment in time, I think the role and function of PCC will likely be reviewed and considered, to determine what serves needs best, and what serves the function of DBL best.
I'm not pointing any conclusions. I don't have one because in many ways this is my first real attempt to understand what this group is talking about wrt ID servers etc.

If that's true, that may be an indication that we need more governance for DBL.  And maybe that's worth discussing with ETEN, given the importance of DBL.

(Which bit of the spec is "hand waving"? Last time I looked it spelled out very clearly how ids are constructed, and all the options are currently in the schema. That seems quite specific to me.)

For DBL content, I can see how to get to a set of unique IDs that do not clash and an overview of what is out there.
For other things, I don't see what that picture looks like yet.  We will have the same need for discoverability.  If we are trying to create sets of materials that translators and scholars can use, it will also be useful to have an overview of who is working on what and how far along it is.
Also, the peripheral flavor gives us an extensibility point, but not a framework for interoperability.  We need a way to specify formats and stylesheets and reference systems and such.  That is very much in the purview of the Copenhagen Alliance.
If all we are doing is defining an archive format with a manifest, then we should stay in our lane.  But even then, we should probably consider the big picture and the associated use cases.   I really think we need a place to gather use cases.

Given recent, tragic events, there are plenty of conversations about DBL governance right now. Whatever the answer to that question, I think it's a completely separate question. Personally, I can't see any management-level team easily committing to supporting all SB flavors, past and future, and I'd expect them to ask questions about mission focus if asked to run a general-purpose registry.

We discussed this in the meeting today and agree that it is an issue that needs discussed, but we noted that it is out of our Scripture Burrito scope.  Specifically, our task is to support interchange between systems.  The details of those systems and the politics of how they came to be and why they currently exist is well beyond our scope (and ability) to control.
Notes from our meeting today record:

Aim for a system that identifies orgs by their FQDN

idServer becomes a declaration that can be shorthand referred to later in the doc
Scope of the prefix is the document itself

Identification section will be modified to not require a whitelist of systemIDs

Scripture Burrito is analogous to EPUB; DBL (and the idea of discoverability) is analogous to Amazon's Kindle ecosystem.
We agree that discoverability is an issue and we are designing SB to support discoverability systems. Note that there is talk of an API system that could help with this (see https://github.com/bible-technology/scripture-burrito-aware-api).  In the next couple of weeks, I think the documentation and the examples will make clear that SB addresses the concerns raised to the extent that it can (and if not, we can reopen this issue after the beta milestone).
I suggest moving the discussion to https://technology.bible/ to engage a wider audience.

authors: jag3773 jonathanrobie klassenjm mvahowe

repo: Copenhagen-Alliance/scripture-burrito


text:
added the env

env  file added

yes checked

authors: Copublica ItzmeAkash

repo: Copublica/HeartBreakBot