text:
Inter-contract recursion depth check

When a cairo program calls the  library_call or call_contract syscalls to invoke another contract, that other contract code runs in a new execution environment, but the recursion depth should be limited and the syscall return an error when it is exceeded.
The current limit on starknet is 50, but this should be verified and also configurable.
c.f #261

I think Inter-contract recursion depth should ideally be checked externally since we're returning control to the user (via the trait) and it's the user's responsibility to make the actual. Any necessary information to implement this should be provided by the user via the trait implementation, so we shouldn't need to do anything in this project.

I agree with what @azteca1998
The power to call another contract via library_call or call_contract lies in the end user, and it's their responsability whether to call another contract or not, they can easily keep a recursion counter in the syscall handler.

authors: azteca1998 edg-l igaray


text:
feat: scarb-native-test

Partially solves #599
Checklist

[x] Linked to Github Issue
[x] Unit tests added
[ ] Integration tests added.
[x] This change requires new documentation.

[x] Documentation has been added/updated.

Codecov Report
Attention: Patch coverage is 87.50000% with 199 lines in your changes are missing coverage. Please review.

Project coverage is 91.37%. Comparing base (511faf7) to head (73e6aa7).

Files
Patch %
Lines

src/bin/utils/test.rs
89.25%
124 Missing :warning:

src/bin/scarb-native-test.rs
0.00%
55 Missing :warning:

src/bin/utils/mod.rs
95.44%
17 Missing :warning:

src/bin/cairo-native-test.rs
70.00%
3 Missing :warning:

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #615      +/-   ##
==========================================
- Coverage   91.38%   91.37%   -0.01%
==========================================
  Files         109      112       +3
  Lines       35889    35952      +63
==========================================
+ Hits        32797    32852      +55
- Misses       3092     3100       +8

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

source: gharchive/pull-request

license_type: permissive

license_source: github-api


text:
0.13.1.1 changes

Update constants
Change Declare fee
Change calldata limit
Check any other changes in the new blockifier version https://github.com/starkware-libs/blockifier/releases/tag/v0.6.0-rc.0

Please can you further explain issue, i want to have an understand of it, so i can take it

@juanbono there are three files (versioned_constants.json, versioned_constants_13_0.json, versioned_constants_13_1.json) in the folder that the link you tagged opened. which file are we to use for the value of constants? Also where are declare fee and calldata limit declared/defined?

source: gharchive/issue

authors: juanbono mubarak23 shyamjayakannan


text:
Unable to build Toybox with Android NDK

I followed instructions for building Toybox with Android NDK.
The newest version I was able to build with NDK (21-25 with API 29-33) was 0.8.7.
Strangely 0.8.5 wouldn't build while 0.8.4 and 0.8.3 did just fine.
Anyone had better luck?
This what I used to build it:
#!/bin/sh -x

#unzip android-ndk-r25c-linux-x86_64.zip
NDK_TC_DIR=~/Android/Sdk/ndk/25.2.9519653/toolchains/llvm/prebuilt/linux-x86_64/bin
NDK_TC=aarch64-linux-android-

cd ${NDK_TC_DIR}
test -f ${NDK_TC}cc || ln -s aarch64-linux-android33-clang ${NDK_TC}cc
test -f ${NDK_TC}strip || ln -s llvm-objcopy ${NDK_TC}strip
cd -

export PATH="${NDK_TC_DIR}:$PATH"

make distclean
make LDFLAGS=--static CROSS_COMPILE=${NDK_TC} defconfig toybox

Below is the failed build output of 0.8.9 (0.8.5+ fail with similar errors):
+ make distclean
cleaned
root cleaned
removed .config
+ make LDFLAGS=--static CROSS_COMPILE=aarch64-linux-android- defconfig toybox
cc -o kconfig/conf kconfig/conf.c kconfig/zconf.tab.c -DKBUILD_NO_NLS=1 \
	-DPROJECT_NAME=\"ToyBox\"
In file included from kconfig/zconf.tab.c:2340:
kconfig/confdata.c: In function ‘conf_write’:
kconfig/confdata.c:435:34: warning: ‘.tmpconfig.’ directive writing 11 bytes into a region of size between 1 and 128 [-Wformat-overflow=]
  435 |                 sprintf(tmpname, "%s.tmpconfig.%d", dirname, (int)getpid());
      |                                  ^~~~~~~~~~~~~~~~~
kconfig/confdata.c:435:17: note: ‘sprintf’ output between 13 and 150 bytes into a destination of size 128
  435 |                 sprintf(tmpname, "%s.tmpconfig.%d", dirname, (int)getpid());
      |                 ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
scripts/genconfig.sh
kconfig/conf -D /dev/null Config.in > /dev/null
scripts/make.sh
Library probe
generated/{Config.in,newtoys.h,flags.h,globals.h,tags.h,help.h}
Compile toybox
..........................toys/lsb/passwd.c:71:45: error: implicit declaration of function 'getspnam' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
  if (*(pass = pw->pw_passwd)=='x' && (sp = getspnam(name))) pass = sp->sp_pwdp;
                                            ^
toys/lsb/passwd.c:71:43: warning: incompatible integer to pointer conversion assigning to 'struct spwd *' from 'int' [-Wint-conversion]
  if (*(pass = pw->pw_passwd)=='x' && (sp = getspnam(name))) pass = sp->sp_pwdp;
                                          ^ ~~~~~~~~~~~~~~
toys/lsb/passwd.c:71:71: error: incomplete definition of type 'struct spwd'
  if (*(pass = pw->pw_passwd)=='x' && (sp = getspnam(name))) pass = sp->sp_pwdp;
                                                                    ~~^
toys/lsb/passwd.c:58:10: note: forward declaration of 'struct spwd'
  struct spwd *sp;
         ^
toys/lsb/passwd.c:93:14: error: implicit declaration of function 'crypt' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
      pass = crypt(toybuf+2048, pw->pw_passwd);
             ^
toys/lsb/passwd.c:93:12: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion]
      pass = crypt(toybuf+2048, pw->pw_passwd);
           ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
toys/lsb/passwd.c:103:17: error: implicit declaration of function 'crypt' is invalid in C99 [-Werror,-Wimplicit-function-declaration]
    encrypted = crypt(toybuf, salt);
                ^
toys/lsb/passwd.c:103:15: warning: incompatible integer to pointer conversion assigning to 'char *' from 'int' [-Wint-conversion]
    encrypted = crypt(toybuf, salt);
              ^ ~~~~~~~~~~~~~~~~~~~
3 warnings and 4 errors generated.
.make: *** [Makefile:17: toybox] Error 1

Not make defconfig, make android_defconfig. Some commands don't build under the NDK.
The biggest chunk of them is the shadow password stuff, which I have a patch locally to rewrite (fixes ~5 commands I think) but I've bumped that until next release...

WAI and -- unlike macOS -- not obvious that make defconfig could ever auto-detect cross-compilation for Android...

authors: enh-google eugenesan landley

repo: landley/toybox


text:
🛑 rexingsports.cn is down

In 58fd1dd, rexingsports.cn (https://www.rexingsports.cn) was down:

HTTP code: 567
Response time: 801 ms

Resolved: rexingsports.cn is back up in e549976 after 19 minutes.

repo: lanen/bs-site

license: MIT


text:
🛑 vipgifts.net is down

In 57e95b7, vipgifts.net (https://vipgifts.net) was down:

HTTP code: 567
Response time: 982 ms

Resolved: vipgifts.net is back up in 451e565 after 25 minutes.


text:
🛑 rexingsports.cn is down

In 37cba33, rexingsports.cn (https://www.rexingsports.cn) was down:

HTTP code: 567
Response time: 685 ms

Resolved: rexingsports.cn is back up in 1ebcd68 after 20 minutes.


text:
🛑 rexingsports.cn is down

In 4f859b0, rexingsports.cn (https://www.rexingsports.cn) was down:

HTTP code: 567
Response time: 819 ms

Resolved: rexingsports.cn is back up in d21d2a5.


text:
🛑 vipgifts.net is down

In 1d235ae, vipgifts.net (https://vipgifts.net) was down:

HTTP code: 567
Response time: 720 ms

Resolved: vipgifts.net is back up in 2af1d5a after 19 minutes.


text:
ReflectionException in Container.php line 734: Class web does not exist

    in Container.php line 734
    at ReflectionClass->__construct('web') in Container.php line 734
    at Container->build('web', array()) in Container.php line 629
    at Container->make('web', array()) in Application.php line 697
    at Application->make('web') in Pipeline.php line 126
    at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
    at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32
    at Pipeline->Illuminate\Routing\{closure}(object(Request))
    at call_user_func(object(Closure), object(Request)) in Pipeline.php line 102
    at Pipeline->then(object(Closure)) in Router.php line 726
    at Router->runRouteWithinStack(object(Route), object(Request)) in Router.php line 699
    at Router->dispatchToRoute(object(Request)) in Router.php line 675
    at Router->dispatch(object(Request)) in Router.php line 83
    at Router->dispatch(object(Request)) in Kernel.php line 246
    at Kernel->Illuminate\Foundation\Http\{closure}(object(Request))
    at call_user_func(object(Closure), object(Request)) in Pipeline.php line 52
    at Pipeline->Illuminate\Routing\{closure}(object(Request)) in CheckForMaintenanceMode.php line 44
    at CheckForMaintenanceMode->handle(object(Request), object(Closure))
    at call_user_func_array(array(object(CheckForMaintenanceMode), 'handle'), array(object(Request), object(Closure))) in Pipeline.php line 136
    at Pipeline->Illuminate\Pipeline\{closure}(object(Request))
    at call_user_func(object(Closure), object(Request)) in Pipeline.php line 32
    at Pipeline->Illuminate\Routing\{closure}(object(Request))
    at call_user_func(object(Closure), object(Request)) in Pipeline.php line 102
    at Pipeline->then(object(Closure)) in Kernel.php line 132
    at Kernel->sendRequestThroughRouter(object(Request)) in Kernel.php line 99
    at Kernel->handle(object(Request)) in index.php line 52

I am getting this error after  applying following

Now open up app/config/app.php and add the service provider to your providers array.
Langaner\RoutePriority\RoutePriorityServiceProvider::class,

Would you please suggest?

Check the web middleware group in your Http\Kernel.php
It must be something like this https://laravel.com/docs/5.2/middleware#middleware-groups

Thanks for the quick reply.
Here is my Http\Kernel.php
<?php namespace App\Http;

use Illuminate\Foundation\Http\Kernel as HttpKernel;
use Langaner;

class Kernel extends HttpKernel {

        use \Langaner\RoutePriority\RouterTrait;
	/**
	 * The application's global HTTP middleware stack.
	 *
	 * @var array
	 */
	protected $middleware = [
        \Illuminate\Foundation\Http\Middleware\CheckForMaintenanceMode::class,
	];

    /**
     * The application's route middleware groups.
     *
     * @var array
     */
    protected $middlewareGroups = [
        'web' => [
            \App\Http\Middleware\EncryptCookies::class,
            \Illuminate\Cookie\Middleware\AddQueuedCookiesToResponse::class,
            \Illuminate\Session\Middleware\StartSession::class,
            \Illuminate\View\Middleware\ShareErrorsFromSession::class,
            \GrahamCampbell\HTMLMin\Http\Middleware\MinifyMiddleware::class,
            \App\Http\Middleware\VerifyCsrfToken::class,
            \App\Http\Middleware\App::class,
        ],
        'api' => [
            'throttle:60,1',
        ],
    ];

	/**
	 * The application's route middleware.
	 *
	 * @var array
	 */
	protected $routeMiddleware = [
        'auth' => \App\Http\Middleware\Authenticate::class,
        'auth.basic' => \Illuminate\Auth\Middleware\AuthenticateWithBasicAuth::class,
        'guest' => \App\Http\Middleware\RedirectIfAuthenticated::class,
		'admin' => \App\Http\Middleware\IsAdmin::class,
		'redac' => \App\Http\Middleware\IsRedactor::class,
		'ajax' => \App\Http\Middleware\IsAjax::class,
            'throttle' => \Illuminate\Routing\Middleware\ThrottleRequests::class,
	];

}

it's look ok as per my thinking
it is working fine if I remove
Langaner\RoutePriority\RoutePriorityServiceProvider::class
from  app/config/app.php
I have also applied
https://github.com/langaner/route-priority/pull/1/commits/95a370672f308be2c8a71ff86595e39dc3088117
still there is not luck.

authors: langaner rohittheindiajobs

repo: langaner/route-priority

license: mit

license_source: bigquery


text:
SQL Agent for Google Big Query

Checked other resources

[X] I added a very descriptive title to this issue.
[X] I searched the LangChain documentation with the integrated search.
[X] I used the GitHub search to find a similar question and didn't find it.
[X] I am sure that this is a bug in LangChain rather than my code.

Example Code
a
Error Message and Stack Trace (if applicable)
a
Description
Is it possible to create an sql agent to make queries on Google BigQuery on latests versions of langchain?. It was possible on older versions.
System Info
a

your example throws the following error:
NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:bigquery

@hugoferrero  your problem is probably with the import. That is my code so far. Can anyone help?
Code
from sqlalchemy import *
from langchain.agents.agent_toolkits import SQLDatabaseToolkit, create_sql_agent
from langchain.sql_database import SQLDatabase
from langchain_google_vertexai import ChatVertexAI

MODEL_NAME = 'codechat-bison'
SQL_ALCHEMY_URL = f'bigquery://{BIGQUERY_PROJECT}/{BIGQUERY_DATASET}?credentials_path={SERVICE_ACCOUNT_KEY_PATH}'
db = SQLDatabase.from_uri(SQL_ALCHEMY_URL)

max_output_tokens = 2048
llm = ChatVertexAI(
    model_name=MODEL_NAME,
    temperature=0,
    max_output_tokens=max_output_tokens
)
toolkit = SQLDatabaseToolkit(db=db, llm=llm)

agent_executor = create_sql_agent(
    llm=llm,
    toolkit=toolkit,
    top_k=1000,  # max. rows
    verbose=True
)
x = ""
agent_executor.invoke(x)

Error Message
_MultiThreadedRendezvous                  Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/google/api_core/grpc_helpers.py](https://localhost:8080/#) in error_remapped_callable(*args, **kwargs)
    161             prefetch_first = getattr(callable_, "_prefetch_first_result_", True)
--> 162             return _StreamingResponseIterator(
    163                 result, prefetch_first_result=prefetch_first

27 frames
_MultiThreadedRendezvous: <_MultiThreadedRendezvous of RPC that terminated with:
	status = StatusCode.INVALID_ARGUMENT
	details = "stop sequence is requested, but model doesn't support;  for stop sequences parameters."
>

The above exception was the direct cause of the following exception:

InvalidArgument                           Traceback (most recent call last)
[/usr/local/lib/python3.10/dist-packages/google/api_core/grpc_helpers.py](https://localhost:8080/#) in error_remapped_callable(*args, **kwargs)
    164             )
    165         except grpc.RpcError as exc:
--> 166             raise exceptions.from_grpc_error(exc) from exc
    167
    168     return error_remapped_callable

InvalidArgument: 400 stop sequence is requested, but model doesn't support;  for stop sequences parameters.

thank you for the response @schimmerd.  I tried with your code a got the same error message. My langchain version is 0.1.6.
Here is the python code:
from sqlalchemy import *
from langchain.agents.agent_toolkits import SQLDatabaseToolkit, create_sql_agent
from langchain.sql_database import SQLDatabase
from langchain.llms import VertexAI

PROJECT_ID = "my_project"
REGION = "my_region"
dataset = "my_dataset"

SQL_ALCHEMY_URL = f'bigquery://{PROJECT_ID}/{dataset}'
db = SQLDatabase.from_uri(SQL_ALCHEMY_URL)

And here is the error message:
NoSuchModuleError                         Traceback (most recent call last)
[/home/flow/Documentos/genai/monetizacion/perfiles/test_langchain_github.py](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/test_langchain_github.py) in line 1
----> 11](file:///home/flow/Documentos/genai/monetizacion/perfiles/test_langchain_github.py?line=10%27%3E11%3C/a%3E) db = SQLDatabase.from_uri(SQL_ALCHEMY_URL)

File [~/anaconda3/envs/llm1/lib/python3.10/site-packages/langchain_community/utilities/sql_database.py:124](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/~/anaconda3/envs/llm1/lib/python3.10/site-packages/langchain_community/utilities/sql_database.py:124), in SQLDatabase.from_uri(cls, database_uri, engine_args, **kwargs)
    122](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/langchain_community/utilities/sql_database.py?line=121%27%3E122%3C/a%3E) """Construct a SQLAlchemy engine from URI."""
    123](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/langchain_community/utilities/sql_database.py?line=122%27%3E123%3C/a%3E) _engine_args = engine_args or {}
--> 124](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/langchain_community/utilities/sql_database.py?line=123%27%3E124%3C/a%3E) return cls(create_engine(database_uri, **_engine_args), **kwargs)

File <string>:2, in create_engine(url, **kwargs)

File [~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py:281](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py:281), in deprecated_params.<locals>.decorate.<locals>.warned(fn, *args, **kwargs)
    274](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=273%27%3E274%3C/a%3E)     if m in kwargs:
    275](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=274%27%3E275%3C/a%3E)         _warn_with_version(
    276](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=275%27%3E276%3C/a%3E)             messages[m],
    277](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=276%27%3E277%3C/a%3E)             versions[m],
    278](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=277%27%3E278%3C/a%3E)             version_warnings[m],
    279](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=278%27%3E279%3C/a%3E)             stacklevel=3,
    280](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=279%27%3E280%3C/a%3E)         )
--> 281](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/deprecations.py?line=280%27%3E281%3C/a%3E) return fn(*args, **kwargs)

File [~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py:552](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py:552), in create_engine(url, **kwargs)
    548](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py?line=547%27%3E548%3C/a%3E) u = _url.make_url(url)
    550](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py?line=549%27%3E550%3C/a%3E) u, plugins, kwargs = u._instantiate_plugins(kwargs)
--> 552](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py?line=551%27%3E552%3C/a%3E) entrypoint = u._get_entrypoint()
    553](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py?line=552%27%3E553%3C/a%3E) _is_async = kwargs.pop("_is_async", False)
    554](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/create.py?line=553%27%3E554%3C/a%3E) if _is_async:

File [~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py:755](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py:755), in URL._get_entrypoint(self)
    753](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=752%27%3E753%3C/a%3E) else:
    754](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=753%27%3E754%3C/a%3E)     name = self.drivername.replace("+", ".")
--> 755](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=754%27%3E755%3C/a%3E) cls = registry.load(name)
    756](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=755%27%3E756%3C/a%3E) # check for legacy dialects that
    757](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=756%27%3E757%3C/a%3E) # would return a module with 'dialect' as the
    758](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=757%27%3E758%3C/a%3E) # actual class
    759](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=758%27%3E759%3C/a%3E) if (
    760](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=759%27%3E760%3C/a%3E)     hasattr(cls, "dialect")
    761](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=760%27%3E761%3C/a%3E)     and isinstance(cls.dialect, type)
    762](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=761%27%3E762%3C/a%3E)     and issubclass(cls.dialect, Dialect)
    763](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/engine/url.py?line=762%27%3E763%3C/a%3E) ):

File [~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py:374](https://file+.vscode-resource.vscode-cdn.net/home/flow/Documentos/genai/monetizacion/perfiles/~/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py:374), in PluginLoader.load(self, name)
    371](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py?line=370%27%3E371%3C/a%3E)         self.impls[name] = impl.load
    372](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py?line=371%27%3E372%3C/a%3E)         return impl.load()
--> 374](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py?line=373%27%3E374%3C/a%3E) raise exc.NoSuchModuleError(
    375](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py?line=374%27%3E375%3C/a%3E)     "Can't load plugin: %s:%s" % (self.group, name)
    376](file:///home/flow/anaconda3/envs/llm1/lib/python3.10/site-packages/sqlalchemy/util/langhelpers.py?line=375%27%3E376%3C/a%3E) )

NoSuchModuleError: Can't load plugin: sqlalchemy.dialects:bigquery

hi @schimmerd
I found a workaround by defining a custom tool, and it worked fine. The react agent now can connect to BigQuery and execute the query. But it seems that "Observation step"  is not working. The query constructed by the model has sintaxis error, and the agent get stuck in a  Question/Thought /Action/Action Input loop. It Can't reach Observation step . Any suggestions about that?
Here is the code with the custom tool:
# google
import vertexai
import google.cloud.bigquery as bq
# Langchain
from langchain_community.utilities import SQLDatabase
from langchain.agents import AgentExecutor, create_react_agent
from langchain_community.agent_toolkits import create_sql_agent
from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit
from langchain_experimental.agents.agent_toolkits.python.base import create_python_agent
from langchain_experimental.tools import PythonREPLTool
from langchain.tools import tool, StructuredTool
from langchain_core.tools import ToolException
from langchain import PromptTemplate
from langchain.llms import VertexAI

# Initialize Vertex AI SDK
vertexai.init(project=project_id, location=region)

#### llm ####
llm = VertexAI(
    model_name="text-bison@002",
    max_output_tokens=2048,
    temperature=0,
    top_p=0.4,
    top_k=40,
    verbose=True)

#### Tools ####

def bq_queries(q: str):
    try:
        client_bq = bq.Client(project=project_id)
        answer = client_bq.query(q).result().to_dataframe()
        return answer
    except Exception as e1:
           exception_str = f"{type(e1).__name__}: {str(e1)}"
    raise ToolException(exception_str)

query= StructuredTool.from_function(
    func=bq_queries,
    name="query",
    description="Executes SQL queries on Google BigQuery and returns a pandas df with the response.",
    handle_tool_error=True
)

tools = [query]

#### Prompt ####
prompt = PromptTemplate(
    input_variables=["input4"],
    template=template_aux_1
)

#### Agent ####

agent = create_react_agent(llm, tools, prompt)

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=True)

agent_executor.invoke({"input4": "¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?"})

And this is the agent behaviour:

```bash
> Entering new AgentExecutor chain...
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table` WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: 437a246b-c039-4cf4-ba77-4b070e1cf7ca
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table`WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: ac2e6252-e2bf-4fbe-8b77-c21e50dcd358
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table` WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: 81263fd4-c351-4be4-abe6-597789a38f1c

import warnings
warnings.filterwarnings('ignore', category=DeprecationWarning)
from google.cloud import bigquery
from google.oauth2 import service_account
from sqlalchemy import create_engine, text
from sqlalchemy.engine.url import make_url, URL
from langchain_community.utilities.sql_database import SQLDatabase
import pandas as pd
from sqlalchemy.exc import SQLAlchemyError
import re
import vertexai
import streamlit as st
Path to your JSON credentials file
service_account_file = r"dev 1.json"  # Change to the path of your service account key file
SQLAlchemy URL for BigQuery
BIGQUERY_PROJECT = ""
BIGQUERY_DATASET = ""
Construct SQLAlchemy URL for BigQuery
sqlalchemy_url = f'bigquery://{BIGQUERY_PROJECT}/{BIGQUERY_DATASET}?credentials_path={service_account_file}'
Create the SQLAlchemy engine
engine = create_engine(sqlalchemy_url)
db = SQLDatabase(engine)
Print available dialects and usable table names
try:
dialects = db.dialect
print(dialects)
except AttributeError as e:
print("AttributeError:", e)
try:
table_names = db.get_usable_table_names()
print("Usable tables:", table_names)
except Exception as e:
print("Error fetching table names:", str(e))
context = db.get_context()
def test_db_connection(db):
try:
table_names = db.get_usable_table_names()
print("Connection successful. Available tables:", table_names)
except SQLAlchemyError as e:
print("Connection failed. Error:", str(e))
Call the function to test the connection
test_db_connection(db)
def execute_sql_query(query: str) -> str:
"""Execute a SQL query and return the results contextually."""
try:
# Check if the input string contains a specific format to extract SQL query
if "query='" in query:
match = re.search(r"query='([^']*)'", query)
if match:
sql_query = match.group(1)
else:
raise ValueError("No SQL query found in the input")
else:
sql_query = query
    with engine.connect() as connection:
        print("Entered SQL execution with db:", sql_query)
        result = connection.execute(text(sql_query))
        rows = result.fetchall()
        columns = result.keys()  # Get the column names
        df = pd.DataFrame(rows, columns=columns)
        st.dataframe(df)
        result_str = ", ".join(str(value) for row in rows for value in row)
        return result_str
except Exception as e:
    st.write("No response available.")  # Display user-friendly message in Streamlit
    return ""

Example usage of execute_sql_query
execute_sql_query("SELECT * FROM your_table")
File "lib\sqlalchemy\cyextension\immutabledict.pyx", line 16, in sqlalchemy.cyextension.immutabledict._immutable_fn
TypeError: immutabledict object is immutable

Thank you. Already solve it by creating a custom tool

hi @schimmerd
I found a workaround by defining a custom tool, and it worked fine. The react agent now can connect to BigQuery and execute the query. But it seems that "Observation step" is not working. The query constructed by the model has sintaxis error, and the agent get stuck in a Question/Thought /Action/Action Input loop. It Can't reach Observation step . Any suggestions about that? Here is the code with the custom tool:
# google
import vertexai
import google.cloud.bigquery as bq
# Langchain
from langchain_community.utilities import SQLDatabase
from langchain.agents import AgentExecutor, create_react_agent
from langchain_community.agent_toolkits import create_sql_agent
from langchain_community.agent_toolkits.sql.toolkit import SQLDatabaseToolkit
from langchain_experimental.agents.agent_toolkits.python.base import create_python_agent
from langchain_experimental.tools import PythonREPLTool
from langchain.tools import tool, StructuredTool
from langchain_core.tools import ToolException
from langchain import PromptTemplate
from langchain.llms import VertexAI

# Initialize Vertex AI SDK
vertexai.init(project=project_id, location=region)

#### llm ####
llm = VertexAI(
    model_name="text-bison@002",
    max_output_tokens=2048,
    temperature=0,
    top_p=0.4,
    top_k=40,
    verbose=True)

#### Tools ####

def bq_queries(q: str):
    try:
        client_bq = bq.Client(project=project_id)
        answer = client_bq.query(q).result().to_dataframe()
        return answer
    except Exception as e1:
           exception_str = f"{type(e1).__name__}: {str(e1)}"
    raise ToolException(exception_str)

query= StructuredTool.from_function(
    func=bq_queries,
    name="query",
    description="Executes SQL queries on Google BigQuery and returns a pandas df with the response.",
    handle_tool_error=True
)

tools = [query]

#### Prompt ####
prompt = PromptTemplate(
    input_variables=["input4"],
    template=template_aux_1
)

#### Agent ####

agent = create_react_agent(llm, tools, prompt)

agent_executor = AgentExecutor(agent=agent, tools=tools, verbose=True, handle_parsing_errors=True)

agent_executor.invoke({"input4": "¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?"})

And this is the agent behaviour:
> Entering new AgentExecutor chain...
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table` WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: 437a246b-c039-4cf4-ba77-4b070e1cf7ca
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table`WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: ac2e6252-e2bf-4fbe-8b77-c21e50dcd358
 Question: ¿Cómo se distribuye, por género y de manera porcentual, la categoría automotor en el periodo 202312?

Thought: Necesito agrupar los datos por género y calcular el porcentaje de usuarios que tienen un nivel de navegación en la categoría automotor mayor a 0.

Action: queries
Action Input: ```
SELECT genero, count(automotor_nivel), ROUND((count(automotor_nivel)/ SUM(count(automotor_nivel)) over ())*100,2)
   FROM `my_project.my_dataset.my_table` WHERE periodo = 202312 AND automotor_nivel > 0 GROUP BY genero;
```BadRequest: 400 Syntax error: Unexpected identifier `` at [1:1]; reason: invalidQuery, location: query, message: Syntax error: Unexpected identifier `` at [1:1]

Location: us-east4
Job ID: 81263fd4-c351-4be4-abe6-597789a38f1c

Could you provide the strings for template_aux_1?, I am trying to replicate with some custom template, but it din't work. Thank you in advance!

authors: Ejananee hugoferrero schimmerd vhcg77

repo: langchain-ai/langchain


text:
Concise API

Feature request
The concise API provides many one-liners for common use cases
Motivation

Many devs are too busy to learn langchain's abstractions and paradigms
Many devs just want concise, ready-to-go LLM tools https://twitter.com/abacaj/status/1654573048912048130?s=20

Your contribution
I have implemented the langchain.concise submodule which contains functions and classes for quickly building language models with minimal code.
The submodule includes the following modules:

choice.py which provides a function for choosing an option from a list of options based on a query and examples.
chunk.py which splits text into smaller chunks.
config.py which provides functions for setting and getting default values for the language model, text splitter, and maximum tokens.
decide.py which determines whether a statement is true or false based on a query and examples.
function.py which defines a decorator for creating reusable text generation functions.
generate.py which generates text using a language model and provides options for removing quotes and retrying failed attempts.
rulex.py which provides a class for defining natural language replacement rules for text.

These modules contain functions that can be used to quickly create language models with minimal code.

https://twitter.com/JacobFValdez

Hi. I made a pr that implements it. Not sure if still relevant but may be good inspiration

Hi, @JacobFV! I'm Dosu, and I'm here to help the LangChain team manage their backlog. I wanted to let you know that we are marking this issue as stale.
From what I understand, you opened this issue requesting a "Concise API" submodule for the langchain repository. The submodule would provide functions and classes for quickly building language models with minimal code. However, there hasn't been any further activity on this issue since you opened it.
Could you please let us know if this issue is still relevant to the latest version of the LangChain repository? If it is, please comment on the issue to let us know. Otherwise, feel free to close the issue yourself, or it will be automatically closed in 7 days.
Thank you for your understanding and contribution to the LangChain project!

Hi. See https://github.com/JacobFV/langchain/commit/b1dcc51ffcb939c53c99345ebd1736e54b7b7545

Closing


text:
Update CONTRIBUTING.md

fiixed few typos

thanks @NinoRisteski!

authors: NinoRisteski baskaryan


text:
Community: Updating Azure Retriever and Docs to be Azure AI Search instead of Azure Cognitive Search

Last year Microsoft changed the name of Azure Cognitive Search to Azure AI Search. This PR updates the Langchain Azure Retriever API and it's associated docs to reflect this change. It may be confusing for users to see the name Cognitive here and AI in the Microsoft documentation which is why this is needed. I've also added a more detailed example to the Azure retriever doc page.
There are more places that need a similar update but I'm breaking it up so the PRs are not too big 😄
Twitter: @marlene_zw
No new tests added.

Thanks for this!
This will break things for users of AzureCognitiveSearchRetriever. Could we keep those modules around, but just have them import AzureAISearchRetriever? Then we can deprecate the old one gracefully if needed.
from langchain_community.retriever.azure_ai_search import AzureAISearchRetriever

# For backwards compatibility
AzureCognitiveSearchRetriever = AzureAISearchRetriever

This assumes that there's no functional differences between the two.
Feel free to remove the draft status of the PR if it is good to go from your perspective.

Thanks for this!
This will break things for users of AzureCognitiveSearchRetriever. Could we keep those modules around, but just have them import AzureAISearchRetriever? Then we can deprecate the old one gracefully if needed.
from langchain_community.retriever.azure_ai_search import AzureAISearchRetriever

# For backwards compatibility
AzureCognitiveSearchRetriever = AzureAISearchRetriever

This assumes that there's no functional differences between the two.
Feel free to remove the draft status of the PR if it is good to go from your perspective.

Makes sense to me @ccurme, I'll make changes and remove draft status once ready to review!

Ooof, sorry, I just pulled from upstream and thought I rebased correctly. Going to close this and start a new PR 😓

authors: ccurme marlenezw


text:
Amazon Bedrock streaming output

Add streaming output for Amazon Bedrock

@sunbc0120
Thanks for adding this to the Bedrock LLM, would you want to wait for #8017 to be merged before adding this? Also, would you want to add an example of usage in the sample notebook?

@sunbc0120 Thanks for adding this to the Bedrock LLM, would you want to wait for #8017 to be merged before adding this? Also, would you want to add an example of usage in the sample notebook?

example notebook: adding
wait for #8017 , don't mind wait, but upon checking the changes, maybe it makes sense it updates after this one merged as it says

raise NotImplementedError(
            """Bedrock doesn't support stream requests at the moment."""
        )

This won't be valid after this PR merged.

Doesn't this also need a streaming boolean flag and changes to the _call to handle streaming?

Is there any documentation on this? It was also my 1st thoughts but failed to find any explanation.

Not just _call for llms but also _stream for the chat model class.

Working on with #10393


text:
Add Self-Hosted Script details to SH Docs

Intitial version with changes.

Promoting to full PR.  No comments on build, so assuming this is ready to go.

repo: langchain-ai/langsmith-docs


text:
[Feature Request] Show Prompt vs. Completion Tokens per Run

Current Behavior
Currently, the LangSmith UI appears to only allow showing prompt/completion tokens for an entire project.

The only view on this data that is available per run is total tokens, but not prompt tokens vs. completion tokens.

Desired Behavior
I'd like to be able to show columns for prompt/completion tokens per run, when viewing a project.

Hi @dmyoung9 - you can see the prompt vs. completion tokens in the run tree once you click on a run

I know you can see it there, but I'd like to be able to see them as columns in the overall view, rather than clicking into every run individually.

Sure will make a ticket

@hinthornw I would also like to have this feature.

@shub-kris and @dmyoung9 , the separation is now visible upon hover in the project table

I'll close this issue for now

Yep, looks good enough to me 👌

authors: dmyoung9 hinthornw shub-kris

repo: langchain-ai/langsmith-sdk


text:
As of openai>=1.0.0, Azure endpoints should be specified via the azure_endpoint param not openai_api_base (or alias base_url). (type=value_error)

As of openai>=1.0.0, Azure endpoints should be specified via the azure_endpoint param not openai_api_base (or alias base_url). (type=value_error)

solution
llm = AzureChatOpenAI(
            temperature=0,
            deployment_name=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],
            openai_api_base=os.environ["AZURE_OPENAI_API_BASE"],
            openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"],
            openai_api_key=os.environ["AZURE_OPENAI_API_KEY"],
            streaming=True,
        )

to
llm = AzureChatOpenAI(
            temperature=0,
            deployment_name=os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"],
            azure_endpoint=os.environ["AZURE_OPENAI_API_BASE"],
            openai_api_version=os.environ["AZURE_OPENAI_API_VERSION"],
            openai_api_key=os.environ["AZURE_OPENAI_API_KEY"],
            streaming=True,
        )

Thanks for input but where to adapt this code (which file)? Or can we expect new release to download soon?

@stevenbaert i'm not the developer of OpenGPTs lol. Since submitting a Pull Request requires approval from OpenGPTs developers, and I have not received approval yet, I guess we should wait.

file: backend/app/llms.py
def get_openai_llm(gpt_4: bool = False, azure: bool = False):

I am getting the same error now even though my api looks exactly as above, using azure_endpoint. I do not have any environment variables set for open_api_base.

authors: mrochon stevenbaert world4jason

repo: langchain-ai/opengpts


text:
An error occurs when calling flowasTool using ToolCallAagent

Bug Description

Reproduction
An error occurs when calling flowasTool using ToolCallAagent
Expected behavior
An error occurs when calling flowasTool using ToolCallAagent
Who can help?
No response
Operating System
windows
Langflow Version
1.0.18
Python Version
3.11
Screenshot
No response
Flow File
No response

I'm sure I added {input} to the ToolCallAgent component promnt

hi @alex-ge123,
can you please provide the JSON file with your flows (the main flow you want to run and the others you want to use as a Tool).
this way I can debug here to understand what can be causing the error for you.
thanks!

@alex-ge123
Thank you for your contribution! This issue will be closed. If you have any questions or encounter another problem, please open a new issue and we will be ready to help you.

authors: Cristhianzl alex-ge123 carlosrcoelho

repo: langflow-ai/langflow


text:
Add Spider Web Scraper & Crawler

Add Spider, the fastest open source scraper & crawler that returns LLM-ready data.
Twitter: @WilliamEspegren

Screencast showing how it works
Screencast from 2024-06-29 14-50-38.webm

Wow. This is nice, @WilliamEspegren
Thank you! I'll approve it ASAP.

Have you seen our new way of building components?

Have you seen our new way of building components?

No, I assume this is not the way since you are mentioning it?

Take a look at the OpenAIModelComponent:
https://github.com/langflow-ai/langflow/blob/f06657d985df07a2299cfa6f4c9f56ab4b39880b/src/backend/base/langflow/components/models/OpenAIModel.py#L23

@ogabrielluiz I actually did that first: https://github.com/langflow-ai/langflow/pull/2439/commits/bbd7136e98693b9a8df14cfa1415c52c837da264 but I got errors

@ogabrielluiz
This:
from typing import Optional
from spider.spider import Spider
from langflow.custom import CustomComponent
from langflow.schema import Data
from langflow.base.langchain_utilities.spider_constants import MODES
from langflow.inputs import (
    SecretStrInput,
    StrInput,
    DropdownInput,
    IntInput,
    BoolInput,
    DictInput,
)

class SpiderTool(CustomComponent):
    display_name: str = "Spider Web Crawler & Scraper"
    description: str = "Spider API for web crawling and scraping."
    output_types: list[str] = ["Document"]
    documentation: str = "https://spider.cloud/docs/api"

    inputs = [
        SecretStrInput(
            name="spider_api_key",
            display_name="Spider API Key",
            required=True,
            password=True,
            info="The Spider API Key, get it from https://spider.cloud",
        ),
        StrInput(
            name="url",
            display_name="URL",
            required=True,
            info="The URL to scrape or crawl",
        ),
        DropdownInput(
            name="mode",
            display_name="Mode",
            required=True,
            options=MODES,
            value=MODES[0],
            info="The mode of operation: scrape or crawl",
        ),
        IntInput(
            name="limit",
            display_name="Limit",
            info="The maximum amount of pages allowed to crawl per website. Set to 0 to crawl all pages.",
            advanced=True,
        ),
        IntInput(
            name="depth",
            display_name="Depth",
            info="The crawl limit for maximum depth. If 0, no limit will be applied.",
            advanced=True,
        ),
        StrInput(
            name="blacklist",
            display_name="Blacklist",
            info="Blacklist paths that you do not want to crawl. Use Regex patterns.",
            advanced=True,
        ),
        StrInput(
            name="whitelist",
            display_name="Whitelist",
            info="Whitelist paths that you want to crawl, ignoring all other routes. Use Regex patterns.",
            advanced=True,
        ),
        BoolInput(
            name="use_readability",
            display_name="Use Readability",
            info="Use readability to pre-process the content for reading.",
            advanced=True,
        ),
        IntInput(
            name="request_timeout",
            display_name="Request Timeout",
            info="Timeout for the request in seconds.",
            advanced=True,
        ),
        BoolInput(
            name="metadata",
            display_name="Metadata",
            info="Include metadata in the response.",
            advanced=True,
        ),
        DictInput(
            name="params",
            display_name="Additional Parameters",
            info="Additional parameters to pass to the API. If provided, other inputs will be ignored.",
        ),
    ]

    def build(
        self,
        spider_api_key: str,
        url: str,
        mode: str,
        limit: Optional[int] = 0,
        depth: Optional[int] = 0,
        blacklist: Optional[str] = None,
        whitelist: Optional[str] = None,
        use_readability: Optional[bool] = False,
        request_timeout: Optional[int] = 30,
        metadata: Optional[bool] = False,
        params: Optional[Data] = None,
    ) -> Data:
        if params:
            parameters = params.__dict__['data']
        else:
            parameters = {
                "limit": limit,
                "depth": depth,
                "blacklist": blacklist,
                "whitelist": whitelist,
                "use_readability": use_readability,
                "request_timeout": request_timeout,
                "metadata": metadata,
                "return_format": "markdown",
            }

        app = Spider(api_key=spider_api_key)
        try:
            if mode == "scrape":
                parameters["limit"] = 1
                result = app.scrape_url(url, parameters)
            elif mode == "crawl":
                result = app.crawl_url(url, parameters)
            else:
                raise ValueError(f"Invalid mode: {mode}. Must be 'scrape' or 'crawl'.")
        except Exception as e:
            raise Exception(f"Error: {str(e)}")

        records = []

        for record in result:
            records.append(Data(data={"content": record["content"], "url": record["url"]}))
        return records

When trying to use the component, I get the following error:
"Error building Component Spider Web Crawler & Scraper: Base type component not found."

@ogabrielluiz any idea why the code in the comment above fails?

@ogabrielluiz
This:
from typing import Optional
from spider.spider import Spider
from langflow.custom import CustomComponent
from langflow.schema import Data
from langflow.base.langchain_utilities.spider_constants import MODES
from langflow.inputs import (
    SecretStrInput,
    StrInput,
    DropdownInput,
    IntInput,
    BoolInput,
    DictInput,
)

class SpiderTool(CustomComponent):
    display_name: str = "Spider Web Crawler & Scraper"
    description: str = "Spider API for web crawling and scraping."
    output_types: list[str] = ["Document"]
    documentation: str = "https://spider.cloud/docs/api"

    inputs = [
        SecretStrInput(
            name="spider_api_key",
            display_name="Spider API Key",
            required=True,
            password=True,
            info="The Spider API Key, get it from https://spider.cloud",
        ),
        StrInput(
            name="url",
            display_name="URL",
            required=True,
            info="The URL to scrape or crawl",
        ),
        DropdownInput(
            name="mode",
            display_name="Mode",
            required=True,
            options=MODES,
            value=MODES[0],
            info="The mode of operation: scrape or crawl",
        ),
        IntInput(
            name="limit",
            display_name="Limit",
            info="The maximum amount of pages allowed to crawl per website. Set to 0 to crawl all pages.",
            advanced=True,
        ),
        IntInput(
            name="depth",
            display_name="Depth",
            info="The crawl limit for maximum depth. If 0, no limit will be applied.",
            advanced=True,
        ),
        StrInput(
            name="blacklist",
            display_name="Blacklist",
            info="Blacklist paths that you do not want to crawl. Use Regex patterns.",
            advanced=True,
        ),
        StrInput(
            name="whitelist",
            display_name="Whitelist",
            info="Whitelist paths that you want to crawl, ignoring all other routes. Use Regex patterns.",
            advanced=True,
        ),
        BoolInput(
            name="use_readability",
            display_name="Use Readability",
            info="Use readability to pre-process the content for reading.",
            advanced=True,
        ),
        IntInput(
            name="request_timeout",
            display_name="Request Timeout",
            info="Timeout for the request in seconds.",
            advanced=True,
        ),
        BoolInput(
            name="metadata",
            display_name="Metadata",
            info="Include metadata in the response.",
            advanced=True,
        ),
        DictInput(
            name="params",
            display_name="Additional Parameters",
            info="Additional parameters to pass to the API. If provided, other inputs will be ignored.",
        ),
    ]

    def build(
        self,
        spider_api_key: str,
        url: str,
        mode: str,
        limit: Optional[int] = 0,
        depth: Optional[int] = 0,
        blacklist: Optional[str] = None,
        whitelist: Optional[str] = None,
        use_readability: Optional[bool] = False,
        request_timeout: Optional[int] = 30,
        metadata: Optional[bool] = False,
        params: Optional[Data] = None,
    ) -> Data:
        if params:
            parameters = params.__dict__['data']
        else:
            parameters = {
                "limit": limit,
                "depth": depth,
                "blacklist": blacklist,
                "whitelist": whitelist,
                "use_readability": use_readability,
                "request_timeout": request_timeout,
                "metadata": metadata,
                "return_format": "markdown",
            }

        app = Spider(api_key=spider_api_key)
        try:
            if mode == "scrape":
                parameters["limit"] = 1
                result = app.scrape_url(url, parameters)
            elif mode == "crawl":
                result = app.crawl_url(url, parameters)
            else:
                raise ValueError(f"Invalid mode: {mode}. Must be 'scrape' or 'crawl'.")
        except Exception as e:
            raise Exception(f"Error: {str(e)}")

        records = []

        for record in result:
            records.append(Data(data={"content": record["content"], "url": record["url"]}))
        return records

When trying to use the component, I get the following error:
"Error building Component Spider Web Crawler & Scraper: Base type component not found."

You should inherit from Component. The import should be from langflow.custom import Component

@ogabrielluiz just bringing this to your attention :)

Hey @WilliamEspegren
You have to set the outputs as well. Check here for an example: https://github.com/langflow-ai/langflow/blob/a6f128c4cfd5b732182435f7f1df8b5ed5e6ce64/src/backend/base/langflow/components/helpers/Memory.py#L60

Screencast from 2024-07-09 20-40-55.webm
    outputs = [
            Output(display_name="Markdown", name="content", method="build"),
            Output(display_name="URL", name="url", method="build"),
    ]

When I have the "outputs" above, the component doesn't even show up. When I comment out the "outputs" the component shows up, but there is no outputs on the node. @ogabrielluiz

Should I solve the merge conflicts?

@ogabrielluiz just pinging for attention

authors: WilliamEspegren ogabrielluiz


text:
关于RestClient::RequestTimeout请求超时

关于请求超时问题，想请问下得如何排除，设置了timeout也不能很好地解决，服务器配置低容易timeout么？

捕捉Timeout的异常，然后做while。

Timeout 设置较长是否可行，目前设置还不算太长，while 相当于死循环了 @lanrion

Timeout你设置长点试试；
while是不是死循环，由你代码决定，搞个计数器，每次重新while sleep一下试试。
请求超时，难道你的是国外的VPS？

Timeout你设置长点试试；
while是不是死循环，由你代码决定，搞个计数器，每次重新while sleep一下试试。
请求超时，难道你的是国外的VPS？

思路大概了解了，就是捕捉异常，失败重试。
阿里云，不过机器配置比较低，1CPU，1G内存，5MB带宽  @lanrion

另外总是会在一些奇怪的地方报错，是服务器的问题还是别的什么问题，cpu率，内存也够用的 @lanrion
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:35:in `set_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:25:in `refresh_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/object_store.rb:16:in `refresh_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:29:in `jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/object_store.rb:11:in `jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/client.rb:48:in `get_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/client.rb:55:in `get_jssign_package'

为什么提问题的人，都喜欢只贴一部分的错误信息，然而并没有什么用。
请提供完整的错误信息。

不好意思，可能上下文信息有点少，跑了3个thin在上面，cpu率不高，内存通过vmstat 查看，swap并未使用，用ab测过并发ab -n 800 -c 250 ，fail 为0，都能成功。主要是有时候成功，有时候报错，这个比较麻烦。 @lanrion
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:35:in `set_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:25:in `refresh_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/object_store.rb:16:in `refresh_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/store.rb:29:in `jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/js_ticket/object_store.rb:11:in `jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/client.rb:48:in `get_jsticket'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/weixin_authorize-1.6.3/lib/weixin_authorize/client.rb:55:in `get_jssign_package'
/home/wwwroot/wexin_app/releases/231/app/controllers/wechat/orders_controller.rb:148:in `buy'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/implicit_render.rb:4:in `send_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/abstract_controller/base.rb:198:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/rendering.rb:10:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/abstract_controller/callbacks.rb:20:in `block in process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:117:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:117:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `block in halting'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:234:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:234:in `block in halting'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:234:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:234:in `block in halting'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `block in halting'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:169:in `block in halting'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:151:in `block in halting_and_conditional'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:92:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:92:in `_run_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:734:in `_run_process_action_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/abstract_controller/callbacks.rb:19:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/rescue.rb:29:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/instrumentation.rb:31:in `block in process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/notifications.rb:164:in `block in instrument'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/notifications/instrumenter.rb:20:in `instrument'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/notifications.rb:164:in `instrument'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/instrumentation.rb:30:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/params_wrapper.rb:250:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.0/lib/active_record/railties/controller_runtime.rb:18:in `process_action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/abstract_controller/base.rb:137:in `process'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionview-4.2.0/lib/action_view/rendering.rb:30:in `process'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal.rb:195:in `dispatch'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal/rack_delegation.rb:13:in `dispatch'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_controller/metal.rb:236:in `block in action'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/routing/route_set.rb:73:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/routing/route_set.rb:73:in `dispatch'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/routing/route_set.rb:42:in `serve'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/journey/router.rb:43:in `block in serve'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/journey/router.rb:30:in `each'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/journey/router.rb:30:in `serve'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/routing/route_set.rb:802:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/config.rb:17:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/etag.rb:24:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/conditionalget.rb:25:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/head.rb:13:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/params_parser.rb:27:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/flash.rb:260:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:225:in `context'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/session/abstract/id.rb:220:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/cookies.rb:560:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.0/lib/active_record/query_cache.rb:36:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activerecord-4.2.0/lib/active_record/connection_adapters/abstract/connection_pool.rb:647:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/callbacks.rb:29:in `block in call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:88:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:88:in `_run_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:734:in `_run_call_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/callbacks.rb:81:in `run_callbacks'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/callbacks.rb:27:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/remote_ip.rb:78:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/debug_exceptions.rb:17:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/show_exceptions.rb:30:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/railties-4.2.0/lib/rails/rack/logger.rb:38:in `call_app'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/railties-4.2.0/lib/rails/rack/logger.rb:20:in `block in call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/tagged_logging.rb:68:in `block in tagged'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/tagged_logging.rb:26:in `tagged'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/tagged_logging.rb:68:in `tagged'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/railties-4.2.0/lib/rails/rack/logger.rb:20:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/actionpack-4.2.0/lib/action_dispatch/middleware/request_id.rb:21:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/methodoverride.rb:22:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/runtime.rb:18:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/activesupport-4.2.0/lib/active_support/cache/strategy/local_cache_middleware.rb:28:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-1.6.4/lib/rack/sendfile.rb:113:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/rack-cors-0.4.0/lib/rack/cors.rb:80:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/railties-4.2.0/lib/rails/engine.rb:518:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/railties-4.2.0/lib/rails/application.rb:164:in `call'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:86:in `block in pre_process'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `catch'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:84:in `pre_process'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:53:in `process'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/connection.rb:39:in `receive_data'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run_machine'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/eventmachine-1.0.7/lib/eventmachine.rb:187:in `run'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/backends/base.rb:73:in `start'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/server.rb:162:in `start'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/controllers/controller.rb:87:in `start'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/runner.rb:200:in `run_command'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/lib/thin/runner.rb:156:in `run!'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/gems/thin-1.6.3/bin/thin:6:in `<top (required)>'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/bin/thin:23:in `load'
/home/wwwroot/wexin_app/releases/231/vendor/bundle/ruby/2.2.0/bin/thin:23:in `<main>'

这段异常还是看不出有什么错误，能不能直接把最顶层的错误信息贴过来？我指的是全部，你可以把敏感信息用其他字符替换。

这已经都是顶层了，Rails.logger.level = 0。
另外调整timeout/open_timeout 到1000，关闭verify_ssl后，似乎有所好转，但是不知是否还有隐患
@lanrion

我也是醉了，貌似加了SSL之后，经常有人问timeout，先去除吧。

是不是有防火墙所以ssl老超时-------- 原始邮件 --------主题：Re: [weixin_authorize] 关于RestClient::RequestTimeout请求超时 (#77)发件人：wikimo 收件人：lanrion/weixin_authorize 抄送：这已经都是顶层了，Rails.logger.level = 0。
另外调整timeout/open_timeout 到1000，关闭verify_ssl后，似乎有所好转，但是不知是否还有隐患
@lanrion
—Reply to this email directly or view it on GitHub.

防火墙确实开了，现在是时好时坏，不稳定，所以感觉就比较难排查 @ChocolateAceCream
关闭ssl后，暂时还算稳定 @lanrion

@wikimo @ChocolateAceCream
这个项目的CI，跑在国外，一般都会有Timeout出现，所以每次本地跑通了，都是直接忽略CI 超时。。

先关闭了。

authors: ChocolateAceCream lanrion wikimo


text:
Project Feedback!

 Congrats on completing the third project!
I have provided a detailed Project 3 Feedback Guide here which covers the most common issues with this submitted project. Read through the feedback guide point-by-point to determine how you could improve your submission.
Let us know if you have any other thoughts or questions about this assignment. Hopefully by now you feel pretty comfortable with all the major pieces to basic Android apps (Views, Controllers, Models, Authentication, API Communication, Preferences, ActionBar, et al) and see how they all fit together.

 Congrats on completing the fourth project!
I have provided a detailed Project 4 Feedback Guide here which covers the most common issues with this submitted project. Read through the feedback guide point-by-point to determine how you could improve your submission.
Let us know if you have any other thoughts or questions about this assignment. Hopefully by now you feel pretty comfortable with all the major pieces to basic Android apps (Views, Controllers, Models, Authentication, API Communication, Preferences, ActionBar, et al) and see how they all fit together.
/cc @codepathreview

repo: laolmstead/SimpleTweet


text:
Unknown option -n

Lapce Version
0.2.0
System information
Linux debian
Describe the bug
$ lapce -V
Lapce 0.2.0
$ lapce -h
Lapce 0.2.0

USAGE:
    lapce [OPTIONS] [PATHS]...

ARGS:
    <PATHS>...

OPTIONS:
    -h, --help       Print help information
    -n, --new
    -V, --version    Print version information
$ lapce -n
Unknown option -n

Additional information
I would expect either the -n flag is not documented, or it doesn't report "unknown option"

can't reproduce

authors: howardjohn panekj

repo: lapce/lapce


text:
Random green patches appear out of nowhere

Lapce Version
0.3.1
System information
Win 11
Describe the bug
Just installed it and some green random patches (screenshot attached) keep appearing as i was exploring it
Additional information

Is your GPU dying?

Its a GTX 1650. A bit outdated now but still works well, never saw this kinda issue with anything before

Try clearing your GPU cache.


text:
.filter and .map methods ?

Hi,
What about .filter and .map methods to manipulate resulted datas?
Your current example
xray('http://mat.io')
  .select(['.Header-list-item a'])
  .run(function(err, array) {
    console.log(array)
    // array is [ 'Github', 'Twitter', 'Lapwing', 'Email' ]
  });

With filter
xray('http://mat.io')
  .select(['.Header-list-item a'])
  .filter(function(e) {
    return e.length < 7
  })
  .run(function(err, array) {
    console.log(array)
    // array is [ 'Github', 'Email' ]
  });

With map
xray('http://mat.io')
  .select(['.Header-list-item a'])
  .map(function(e) {
    return e + '!'
  })
  .run(function(err, array) {
    console.log(array)
    // array is [ 'Github!', 'Twitter!', 'Lapwing!', 'Email!' ]
  });

Useful to manipulate data just before .write('out.json')
It can probably be done with .use(plugin) but I think this kind of method can be helpful in the builtin version
What do you think?

Have you tried .format() ?

Oh yes, format works exactly like map with the 3 arguments. Thanks @olivierlesnicki !
.format(function(elm, idx, arr) {
  return elm + '!'
})

It was not clear for me because the help only said one argument. But also the name .format create confusion. @MatthewMueller perhaps the regular .map name is better here?
.format(function(obj) {
  return mustache('{{title}}', obj);
})

I think filter can be a good friend for format || map

@jeromedecoster what happens if you don't return anything in the .format function?
.format(function(elm, idx, arr) {
  if(e.length > 7) {
    return elm + '!'
  }
})

Well, your code with length > 7 returns
xray('http://mat.io')
  .select(['.Header-list-item a'])
  .format(function(elm, idx, arr) {
    if(elm.length > 7) {
      return elm + '!'
    }
  })
  .run(function(err, array) {
    console.log(array)
    // array is [ undefined, undefined, undefined, undefined ]
  });

But if you mean filter like my previous example with length < 7 it still not works like filter
xray('http://mat.io')
  .select(['.Header-list-item a'])
  .format(function(elm, idx, arr) {
    if(elm.length < 7) {
      return elm + '!'
    }
  })
  .run(function(err, array) {
    console.log(array)
    // array is [ 'Github!', undefined, undefined, 'Email!' ]
  });

@jeromedecoster sorry that's what I meant!
filter would be a nice feature

@jeromedecoster Actually why can't we just do:
xray('http://mat.io')
.select(['.Header-list-item a'])
.format(function(elm, idx, arr) {
return elm + '!'
}
})
.run(function(err, array) {
console.log(array.filter(function (elm) {
if(elm.length < 8) {
return elm;
}
});
// array is [ 'Github!', 'Email!' ]
});

yes I known but
xray('http://mat.io')
.select(['.Header-list-item a'])
.fomat(fn)
.run(function(err, arr) {
  arr = arr.filter(fn)
  fs.writeFileSync('out.json', JSON.stringify(arr))
})

it's not pretty as below.
xray('http://mat.io')
.select(['.Header-list-item a'])
.map(fn)
.filter(fn)
.write('out.json');

I prefer use the filter then write method instead of the run then filter inside run then write file and stringify JSON inside run

@jeromedecoster  Fair point.

I'm +1 to those methods. Just need to make sure it also supports:
xray('http://mat.io')
.select('.Header-list-item a')
.map(fn)
.filter(fn)
.write('out.json');

and:
xray('http://mat.io')
.select({ link: '.Header-list-item a' })
.map(fn)
.filter(fn)
.write('out.json');

This would still be cool to have, I'd accept a PR against 1.0.0 :-)

authors: MatthewMueller jeromedecoster matthewmueller olivierlesnicki

repo: lapwinglabs/x-ray


text:
composer install

why composer install , not add cmd  php artisan key:generate?

Because you really don't want that.

That would break all apps every time you deployed them.

You should never run composer update to update a production app - you should do that first, check it works, commit the new lock file, then run composer install on prod.

authors: GrahamCampbell dota-web

repo: laravel/framework


text:
[5.1] Errors in views cause problems when running in queue daemon worker

I've run into the same problem as described in #9881, so please read that as this follows on, but with more findings, reproduction steps and discussion of a fixe.
As identified in a comment, if an error occurs when rendering a view while in a queue daemon worker, subsequent renderings use a previous rendering result. Seeing as the queue workers are most commonly used for mail and it's easiest to reproduce like that, we will.
I have tested using the beanstalkd driver, but I presume any other than sync will work.
To reproduce:
1, Create an email view using blade @section and @yield, with a method for making it error on demand e.g. trying to echo an undefined variable based on data passed.
2. Send mail to queue using some data that will identify the content as being the 1st email, not triggering error in view.
3. Send mail to queue using some data that will identify the content as being the 2nd email, and triggering error in view.
4. Send mail to queue using some data that will identify the content as being the 3rd email, not triggering error in view.
5. Send mail to queue using some data that will identify the content as being the 4th email, not triggering error in view.
6. Run php artisan queue:work --daemon --tries=1
Expected result:
1st, 3rd and 4th emails received, content correct and different in all three.
Actual result:
1st email correct, 3rd email correct, 4th email has same content as 3rd email.
I've gone beyond what was figured in #9881 and chased this error through the framework and ascertained that the issue is due to the error in the rendering of the view in the 2nd email causing a problem with the section array/section stack in Illuminate\View\Factory. It seems that an error in a view in a previous job effectively get "left over" and causes problems with view rendering in future jobs.
A user-implementable workaround, which also proves the cause, is to add:
Queue::looping(function () {
    View::flushSections();
});

to e.g. the AppServiceProvider boot method. With this in place, the sections are forcibly flushed after every job, and the steps above can be re-run with the expected results achieved.
Exactly how best to approach fixing this in the framework, I am not sure, hence no pull request, but hopefully there is enough information here to be able to replicate this and work to getting it fixed, as this can potentially lead to inadvertently to sending emails with the sensitive information to the wrong recipients.

Got the same issue. This is a MAJOR security leak in Laravel!

Got the same issue. This is a MAJOR security leak in Laravel!

I disagree.

Oh, wait, I see. This is actually a known issue.

On reading again, it's not. I was thinking this was the security bug we still have with @parent.

e.g. trying to echo an undefined variable based on data passed.

This is an error in your code, not ours.

Oh wow. Bugs reported by many people and you close it? Get a fucking grip.
On Tue, 2 Feb 2016 02:45 Graham Campbell notifications@github.com wrote:

On reading again, it's not. I was thinking this was the security bug we
still have with @parent.
e.g. trying to echo an undefined variable based on data passed.
This is an error in your code, not ours.
—
Reply to this email directly or view it on GitHub
https://github.com/laravel/framework/issues/11892#issuecomment-178062371
.

@GrahamCampbell, the problem is not if he has or has not an error in his code.
Yes, he has. Errors happens.
I think you're missing the point here. The problem is not that we have an undefined variable in our code. The problem is how Laravel REACTS to code errors in views (you know, errors get their way in our code from time to time, we're humans).
If the steps mentioned by @ockle are reproducible, to me it seems like a clear Laravel bug.
An error in a view should trigger a fatal error no matter what. The email should not be sent, period.
What does Laravel when sending mails in a queue and gets an error in the view? It still sends the mail, but with the last successful rendered email view. That's insane.
The security implications are HIGH, because you might very well send sensitive data to other users (think of "Welcome user X. Here are your username/password.", sent to user Y.
That's very SERIOUS.
And the worst part? I run into this bug two times already, when the system sent by mistake wrong emails to wrong users.

Proposed fix?

@robclancy, the only way you can get @GrahamCampbell to do anything is if there are extra whitespaces.  Otherwise, he closes the PR early and asks questions later. @taylorotwell one of these days we'd like to not have to deal with him...

Will try to re-create in a bit.

Pushing up fixes.

@jaketoolson I don't appreciate your comments, and it doesn't help when they're not even true.

@GrahamCampbell then do what you always do, close the thread without offering any help.

@taylorotwell I am testing both locally and on our dev servers. Finger crossed as this would aleviate one of the strangest quirks we encountered and ignored :)

OK just let me know as soon as you know :)
On Feb 1, 2016, 2:33 PM -0600, Jake Toolsonnotifications@github.com, wrote:

@taylorotwell(https://github.com/taylorotwell)I am testing both locally and on our dev servers. Finger crossed as this would aleviate one of the strangest quirks we encountered and ignored :)
—
Reply to this email directly orview it on GitHub(https://github.com/laravel/framework/issues/11892#issuecomment-178174107).

@taylorotwell Thanks for looking at this, however, your fix does not seem to solve the issue :(
Here's a some stripped out reproduction steps for a fresh install of L5.2:
routes.php:
/*Queue::looping(function () {
    View::flushSections();
});*/

Route::group(['middleware' => ['web']], function () {
    Route::get('/test', function () {
        Mail::queue('email.test', ['error' => false, 'foo' => 'One'], function ($message) {
            $message->to('test@example.com')
                ->subject('One');
        });
        Mail::queue('email.test', ['error' => true, 'foo' => 'Two'], function ($message) {
            $message->to('test@example.com')
                ->subject('Two');
        });
        Mail::queue('email.test', ['error' => false, 'foo' => 'Three'], function ($message) {
            $message->to('test@example.com')
                ->subject('Three');
        });
        Mail::queue('email.test', ['error' => false, 'foo' => 'Four'], function ($message) {
            $message->to('test@example.com')
                ->subject('Four');
        });
        Mail::queue('email.test', ['error' => false, 'foo' => 'Five'], function ($message) {
            $message->to('test@example.com')
                ->subject('Five');
        });
    });
});

view/email/layout.blade.php:
Layout

@yield('content')

views/email/test.blade.php:
@extends('email.layout')

@section('content')
    {{ $foo }}

    @if ($error)
        {{ $bar }}
    @endif
@endsection

Set mail driver to log. Now visit /test. The run php artisan queue:work --daemon --tries=1. Once that has processed all the jobs, stop it and check the log. The last three emails will all have content "Three". Now uncomment that bit at the top of the posted routes.php and rerun the test. Now last three emails have the correct content.
This is tested with "laravel/framework": "5.2.*@dev" which has pulled in 8116de4640fc67b60cb9c418b5f25b4948d69448 and verified View.php is patched.

Hmm. Anyone see anything I'm missing on what may be causing this?
On Feb 1, 2016, 2:36 PM -0600, ocklenotifications@github.com, wrote:

@taylorotwell(https://github.com/taylorotwell)Thanks for looking at this, however, your fix does not seem to solve the issue :(
Here's a some stripped out reproduction steps for a fresh install of L5.2:
routes.php:
/Queue::looping(function () {View::flushSections();});/Route::group(['middleware'=>['web']],function() {Route::get('/test',function() {Mail::queue('email.test', ['error'=>false,'foo'=>'One'],function($message) {$message->to('test@example.com')->subject('One');});Mail::queue('email.test', ['error'=>true,'foo'=>'Two'],function($message) {$message->to('test@example.com')->subject('Two');});Mail::queue('email.test', ['error'=>false,'foo'=>'Three'],function($message) {$message->to('test@example.com')->subject('Three');});Mail::queue('email.test', ['error'=>false,'foo'=>'Four'],function($message) {$message->to('test@example.com')->subject('Four');});Mail::queue('email.test', ['error'=>false,'foo'=>'Five'],function($message) {$message->to('test@example.com')->subject('Five');});});});
view/email/layout.blade.php:
Layout@yield('content')
views/email/test.blade.php:
@extends('email.layout')@section('content'){{$foo}}@if($error){{$bar}}@endif@endsection
Set mail driver to log. Now visit /test. The runphp artisan queue:work --daemon --tries=1. Once that has processed all the jobs, stop it and check the log. The last three emails will all have content "Three". Now uncomment that bit at the top of the posted routes.php and rerun the test. Now last three emails have the correct content.
This is tested with"laravel/framework": "5.2.*@dev"which has pulled in8116de4640fc67b60cb9c418b5f25b4948d69448and verified View.php is patched.
—
Reply to this email directly orview it on GitHub(https://github.com/laravel/framework/issues/11892#issuecomment-178175923).

I have been thinking a lot about this and the fact that the sections are left over from the last successful email left and not the email that has the exception in it means the issue is occurring before this point. However this makes no sense because firstly it only happens when the exception happens and secondly now when the error does happen the sections are flushed so even if it isn't fixing the root cause it should still be fixing it.
We need this fixed now so will be taking the test code above and working until I find the issue.

What version of PHP are you on?
On Feb 2, 2016, 1:25 AM -0600, Robert Clancy (Robbo)notifications@github.com, wrote:

I have been thinking a lot about this and the fact that the sections are left over from the last successful email left and not the email that has the exception in it means the issue is occurring before this point. However this makes no sense because firstly it only happens when the exception happens and secondly now when the error does happen the sections are flushed so even if it isn't fixing the root cause it should still be fixing it.
We need this fixed now so will be taking the test code above and working until I find the issue.
—
Reply to this email directly orview it on GitHub(https://github.com/laravel/framework/issues/11892#issuecomment-178422217).

This has been fixed and tagged on Laravel 4.2 through 5.2.

@taylorotwell I've just updated to 5.1.23 and run the test I posted previously, and I'm not seeing it as fixed, sorry. If the problem lied in flushSections(), then I wouldn't expect calling it after every loop to have fixed the issue before your most recent patch, which it did (and still does).

Tested with 5.1.29 and can confirm it's fixed, thanks!

Great! Thanks, Taylor!

This had been a frustrating issue. Glad for the fix.

I get the same error when the Swift_Mail class emit error on connection with SMTP Server (timeout). I think this error is about send email using queue and not isolated to blade templates.
Laravel framework version: 5.1.29

Seeing that this is a major security issue and not something necessarily you'd expect, it would be great to know if there were some kind of channel where issues like these get communicated?

but this is not fixed in Laravel Framework version 5.0.35

I'm locking this issue because it either has gone off-topic, become a dumping ground for things which shouldn't be in an issue tracker or is just too old.

authors: GrahamCampbell driesvints jaketoolson lifeofguenter ockle pavinthan pedro-lucas robclancy taylorotwell vladrusu zogot