text:
Updated XML helper to support storing and parsing of Guid

Updated XML helper to support storing and parsing of Guid
@BenedekFarkas

I like this change! We should add some tests though (testing on 1.10.x is now fully functional).

@BenedekFarkas @sebastienros should we merge this? Or wait for the tests Benedek mentions?

I'll add tests in this PR.

repo: OrchardCMS/Orchard


text:
Add a robots.txt file

Good for seo, and needed to user tools like http://browsershots.org/

A robots.txt is definitely needed to avoid such errors.
The fix could be:

to add a robots.txt file to the docs/ folder or possibly create a seperate files/ folder to include it in.
This could be the robots.txt file.

Then finally add the following lines to views/web_views.py:
@app.route('/robots.txt')
def robots():
    return app.send_static_file('docs/robots.txt')  # or app.send_static_file('files/robots.txt')

Let me know what you think before I make any PRs :smiley:

@Zero-1729 Sorry to be so late on this.
Yes, that looks great. There's nothing we need to hide from crawlers.
You want to submit that PR?

@wanderingstan No worries.
Just submitted the PR :shipit:

THanks @Zero-1729 ! Merged and pushed to production.

source: gharchive/issue

repo: OriginProtocol/company-website

license: MIT

license_type: permissive

license_source: github-api


text:
Add Traditional Chinese (zh_tw.json)

Add Traditional Chinese localization

Hey there,
thank you very much for helping us to add support for more languages in our mod.
We appreciate your effort to improve the mod together with us.
We will add this in our next update.
Your Inventive Inventory Team

source: gharchive/pull-request

authors: LU-XD yichifauzi

repo: Origins-Tech/InventiveInventory


text:
Removed some outdated links

Issue #394

Is this contribution valid for hacktoberfest2021?


text:
подключаемся в одну строку

Выполнено ДЗ №

[x] Основное ДЗ
[ ] Задание со *

В процессе сделано:

Пункт 1
Пункт 2

Как запустить проект:

Например, запустить команду X в директории Y

Как проверить работоспособность:

Например, перейти по ссылке http://localhost:8080

PR checklist

[ ] Выставил label с номером домашнего задания
[ ] Выставил label с темой домашнего задания

Здравствуйте!) Проверил ваше дз. К сожалению у вас отсутствуют тесты как таковые. Рекомендую свериться с методичкой из предыдущего дз. ДЗ возвращаю на доработку. (Так же не хватает некоторых файлов)

у меня это было в ветке маин, я ее не подмержил перед отправкой ПР. сейчас подмержил, файлы появились


text:
WPS for querying ECCC weather forecasts through the GeoMET WPS service for the watershed of interest.

Description

Self info:
WCS for netcdf example link:
https://geo.weather.gc.ca/geomet?SERVICE=WCS&VERSION=2.0.1&REQUEST=GetCoverage&COVERAGEID=RDPA.24F_PR&SUBSETTINGCRS=EPSG:4326&SUBSET=x(-180,56.25000000000001)&SUBSET=y(12.211180191503997,85.70184378613162)&FORMAT=image/netcdf&TIME=2020-03-10T12:00:00Z

Replace COVERAGEID with the product (RDPS/GDPS)
Probably want to change the 12Z to 6Z for Canada, if available

Variables description from: https://directory.spatineo.com/service/12371/
ETA_PR --> Precip accum [kg/(m^2)]
ETA_RN --> Rain accum [kg/(m^2)]
ETA_SN --> Snow accum [kg/(m^2)]
ETA_TT --> Air temperature [°C]

@huard I have started the WPS process and test suite but there seems to be an intrinsic limitation in GeoMet:
"Note that queries with time ranges or multiple time values are not currently supported on GeoMet for both TIME or DIM_REFERENCE_TIME parameters. Only single time queries are supported."
What I gather from this is that we can only download one forecast step at a time... not super useful if we want to download 4 variables on say 40 time steps for each of 4 models (RDPS/GDPS/REPS/GEPS). Initial tests have shown that there is no time coordinate in the returned WCS-called netcdf...
see branch "wps_geomet" for details...

@huard @tlogan2000 It might take a while before we can extract large amounts of data in single calls from GeoMET. Can we have a plan B where we download the ECCC forecasts from the datamart every day and store it temporarily on THREDDS (or elsewhere) for the past, say, 3-4 days? The data on Datamart is accessible for full forecasts (all lead-times at once) so that would make things much simpler.
I don't know how complicated it would be to setup an auto-download process for the required files, but if it's possible it's probably the next best thing. Or else we'll have to resort to submitting like 640 GeoMET requests each time we want to do a forecast.
Any comments or suggestions welcome!

Creating the proper download script is probably the jump-off point and the most critical.  Automating it to run every day should be fairly simply for Long I would think....

Creating the proper download script is probably the jump-off point and the most critical. Automating it to run every day should be fairly simply for Long I would think....

Yes.  Someone write the script, I can hook it up to our cronjobs on PAVICS easily.

It would seem ECCC has an "Advanced Message Queuing Protocol" (AMQP) that allows automatically receiving the desired forecast data as soon as it is available: https://eccc-msc.github.io/open-data/msc-datamart/amqp_fr/
It runs in python so should be relatively easy to implement I would think? I know some hydropower utilities use this and usually works well.
Now just trying to find the good variables on the datamart, which seems to be proving more difficult than anticipated...

Closed with #321

authors: huard richardarsenault tlogan2000 tlvu

repo: Ouranosinc/raven


text:
wand_gui doesn’t run properly with python3.8

When running the wand_gui with python3.8, the gui does not update when using any of the gui operations (eg fast mode). However, the gui receives updates when other clients (using other py versions) have the fast mode selected, for example.
Debugging in  wand\gui.py:
The fast mode button works:
print(data) in add_async_cb gives ('fast_mode',)
however this doesn’t reach fast_mode_cb
print(self.fut.done()) is True in  add_async_cb. However, print(self.fut.result()) in add_async_cb gives the following error:
File "/home/ion/scratch/wand_new/wand/gui.py", line 123, in add_async_cb
    print(self.fut.result())
  File "/home/ion/scratch/wand_new/wand/gui.py", line 300, in loop
    await asyncio.wait_for(self.wake_loop.wait(),
  File "/home/ion/anaconda3/envs/wand_py38/lib/python3.8/asyncio/tasks.py", line 448, in wait_for
    loop = events.get_running_loop()
RuntimeError: no running event loop

Summarising slack conversations...the behaviour of wait_for seems to be different in newer versions of python. Replacing it with an await (no timeout) works fine. So it looks like wait_for is complaining that no event loop is running, even though await works fine. This feels like a bug somewhere outside of our code (either asyncio or quamash).
Let's try passing our event loop into wait_for as a kwarg. That will be depreciated in a future python revision, but let's hope the underlying bug has been fixed before that happens.

cf https://github.com/python/cpython/blob/a1652da2c89bb21f3fdc71780b63b1de2dff11f0/Lib/asyncio/tasks.py#L435

I suspect the issue here is that the get_event_loop code here https://github.com/python/cpython/blob/master/Lib/asyncio/events.py is thread specific and this is being called from a coroutine in a qt thread....
@bazavano can you try one thing quickly: at the start of the loop function call set_event_loop passing in the event loop. If that works can you submit a PR to close this issue. Put a comment next to the line of code that references this issue.

Sending the event loop via set_event_loop  at the beginning of the loop function doesn't solve the issue and with print(self.fut.result()) in add_async_cb catches the following error when pressing fast mode:
('fast_mode',)
Traceback (most recent call last):
File "/home/ion/scratch/wand_new/wand/gui.py", line 124, in add_async_cb
print(self.fut.result())
File "/home/ion/scratch/wand_new/wand/gui.py", line 124, in add_async_cb
print(self.fut.result())
File "/home/ion/scratch/wand_new/wand/gui.py", line 124, in add_async_cb
print(self.fut.result())
[Previous line repeated 3 more times]
File "/home/ion/scratch/wand_new/wand/gui.py", line 302, in loop
await asyncio.wait_for(self.wake_loop.wait(), next_measurement_in) #, loop = self._gui.loop)
File "/home/ion/anaconda3/envs/wand_py38/lib/python3.8/asyncio/tasks.py", line 448, in wait_for
loop = events.get_running_loop()
RuntimeError: no running event loop

uurgh...well, let's add the hack in to pass the event loop into wait_for. We can then (a) hop this goes away in some future version of python (prior to that parameter being depreciated) (b) do some debugging inside asyncio to figure out why get_event_loop isn't returning the event loop we've set (but if this gets into some qt thread issue, that could be quite non-trivial so I'm not volunteering right away)...

the loop  parameter in wait_for will be deprecated in py3.10, so this is not a long-term solution, but will do until someone has time to understand the issue better

authors: bazavano hartytp

repo: OxfordIonTrapGroup/wand


text:
Runtime Error on Windows

Built with master branch on Windows 11. Same error w/wo CUDA.
Unhandled exception at 0x00007FFB0C804477 (ggml.dll) in main.exe: 0xC00000FD: Stack overflow (parameters: 0x0000000000000001, 0x0000009026403000).

I have the same issue running a 64 bit binary on Windows 10 compiled with VS2022.  The bark and encodec models were converted with the python scripts to f16.  No errors output when converting the models or running main.exe, it just crashes at this function call.

@jzeiber @MichaelZhangBH Hello both! Thanks for opening this issue.
@MichaelZhangBH Could you post the entire traceback?

Hello @PABannier , great to have you look into this. I will put the call stack here.
 	ggml.dll!__chkstk() Line 109	Unknown
>	ggml.dll!ggml_allocr_new_from_buffer(ggml_backend_buffer * buffer) Line 304	C
 	main.exe!bark_model_load(std::basic_ifstream<char,std::char_traits<char>> & fin, gpt_model & model, int n_gpu_layers, bark_verbosity_level verbosity) Line 882	C++
 	main.exe!bark_load_model_from_file(const std::string & fname, bark_model * model, int n_gpu_layers, bark_verbosity_level verbosity) Line 1029	C++
 	main.exe!bark_load_model(const std::string & model_path, bark_verbosity_level verbosity) Line 1066	C++
 	main.exe!main(int argc, char * * argv) Line 34	C++
 	[External Code]

Also, BTW, is there any plan to upgrade the to some of the more recent commits of ggml.

@MichaelZhangBH Yes, it is on my todo list. I'll try to do that in the next few days!

Hello @PABannier can you please upload proper working binary files for Windows to Release?

I was able to get it working.  The default stack size in Visual Studio is 1MB.  I changed it to 100MB, and now bark successfully generates output.

@jzeiber Thank you for trying! Could you open a PR and add your fix in a CMakeLists.txt file? Using if(MSVC) and increasing the stack size should do the trick.

Closing as this is fixed with #163

authors: MichaelZhangBH PABannier jzeiber progressionnetwork


text:
Remove pandas profiling from profile view

Pandas profiling is a great tool for analyzing a raw data table, but often results in conflicting data types as compared to what is explicitly declared in the EML metadata for the given data table (see https://dex.edirepository.org/https%3A%2F%2Fpasta-d.lternet.edu%2Fpackage%2Fdata%2Feml%2Fedi%2F1%2F1%2Fcba4645e845957d015008e7bccf4f902 as an example). For this reason, pandas profiling should be removed completely from Dex and be replaced with a profile that is rooted in the table description as declared within the EML metadata. A simple head/tail view and describe of numeric columns would also be useful. In addition, perhaps a declared column attribute name and found column attribute view would be helpful.

Corrections to Pandas Profiling integration has now resulted in a much better user experience. I recommend keeping the current integration until demonstrated otherwise.

repo: PASTAplus/dex


text:
Genome Browser: Upload user data

Allow users to upload their own data in standard file formats (BAM, BigWig, GFF, etc) and display them as custom tracks.
Genome browser is an important tool for providing integrated view of the multi-omics data and using comparing experimental data to annotations. We need to preserve the basic functionality that allow users to upload their data directly to genome browser and later provide tighter integration with the data in user workspace.
-Maulik

Fixed.

license: mit

license_source: bigquery


text:
Improve the parallel coordinate plot code

Added parameters to enable label for violin plot in legend and to allow more of customization of the legend box
Minor clean up for less print on screen
Regenerated example plots using the latest code and data

@kristinchang3 thanks! I am merging the PR.


text:
pex & multiverse

hi, I would like to deny permission to set homes in a specific world, what can I do?

Why are you opening a PR lol

repo: PEXPlugins/PermissionsEx


text:
Constraints in Parameters

When defining parameters, a set of constraints could be used. This resambles to Pyro parameters.
In this regard, the following code
    qtheta_scale = tf.math.softplus(models.Parameter(1., name="qtheta_scale"))

might be written as
    qtheta_scale = models.Parameter(1., name="qtheta_scale", constraints=constraints.positive)

This can be implemented with bijectors. As this will be required for MCMC inference in some cases, I suggest to wait and see if a common interface can be implemented for every case where some objects are restricted to some specific universe.

repo: PGM-Lab/InferPy


text:
Namespace would be nice

I mean, it's 2017...

pull request in #47

repo: PHPGangsta/GoogleAuthenticator


text:
[Question] 请问readme的效果对比图是在目前开放的10K数据，和scripts的默认配置跑的吗？

Required prerequisites

[X] I have read the documentation https://safe-rlhf.readthedocs.io.
[X] I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
[X] Consider asking first in a Discussion.

Questions
请问readme的效果对比图是在目前开放的10K数据，和scripts的默认配置跑的吗？
这个结果是safe-rlhf的吧，有没有rlhf的结果呢？
我基于chinese-alpaca-plus-lora-7b训练，因为gpu资源问题，超参数部分和默认的不一样（ epoch, 和batch size），评测rlhf和safe-rlhf是用gpt4自动评测那83个问题。rlhf win, tile, lose = 25:36:22, safe-rlhf 19:28:36。没有你们给的结果那么明显
Checklist

[X] I have provided all relevant and necessary information above.
[X] I have chosen a suitable title for this issue.

超参数部分和默认的不一样（ epoch, 和batch size）

@LiuShixing 感谢提问，我们目前只开源了我们第一轮 RLHF 的数据，今明两天会开源更多数据。RLHF 训练取决于你的 reward model 和 cost model 的数据分布，并需要依此进行超参选择。没有现成的最佳超参数。

请问readme的效果对比图是在目前开放的10K数据

是的

和scripts的默认配置跑的吗？

不是。目前 scripts 上的 threshold（即 $C_{\text{limit}}$）默认值是 0。正常情况下需要根据数据分布和 reward model / cost model 训练情况详细调参。由于不同超参和不同数据集对应的最佳参数不一样，故 scripts 中提供的 threshold 只取了 0。

我基于chinese-alpaca-plus-lora-7b训练，因为gpu资源问题，超参数部分和默认的不一样（ epoch, 和batch size）

考虑到你们即换了 model / tokenizer，还更换了超参数，建议重新进行超参数搜索找到更好的超参。

谢谢

authors: LiuShixing XuehaiPan

repo: PKU-Alignment/safe-rlhf


text:
Ähnliches Problem

Hallo PLCHome, leider weiß ich nicht wie ich am besten Kontakt mit Dir aufnehmen kann und ich bin auch neu in Github, bin aber auf dich gestoßen, weil ich exakt das Problem habe, die Werte meines Growatt Wechselrichters abzugreifen und ich glaube Du könntest mir helfen. Habe versucht über Python mir was zu basteln, komme aber nicht weiter...
Ohne Dich zu belästigen und man kann das hier natürlich auch sofort dann löschen, aber wie kann ich Dich kurz dazu befragen, wo ich die speziellen Werte für meinen Growatt-Account erhalte? Bislang habe ich nur Gesamtwerte wie über response = session.post("https://server.growatt.com/selectPlant/getPlantList", data={"currPage": "1","plantType": "-1","orderType": "2","plantName": "".) Wäre Dir super dankbar, Chrissili

Aktuell nutzt die API /index/getPlantListTitle
https://github.com/PLCHome/growatt/blob/46a5ed43ad5a4676e0a718110c21bf6c74140ab4/lib/growatt.js#L287
Der Code zur Abfrage ist ausgelagert.
Vielleicht kannst du ihn ja übersetzen.
https://github.com/PLCHome/growatt/blob/master/lib/growatt.js

Danke Dir, das hat mir schon sehr weitergeholfen. Aber kommt man mit diesem growatt.js auch an die "live Werte"?  PV Erzeugung, Batterieprozentsatz und Netzverbrauch? oder brauche ich da ein anderes Skript bzw. anderen Pfad? Wäre toll, wenn Du mir da nochmal kurz weiterhelfen könntest...

Also bei meinem SPH4000 funktioniert das mit den Livewerten.
Es gibt zwei Möglichkeiten einmal über die Statusanzeige und über die historischen Daten.
Beim Max ging es nur über die historischen Daten, der gibt die auch in der Webansicht nicht aus, sondern zeigt die Erzeugung nur als Graph an.

Am einfachsten ist es wahrscheinlich, sich die Daten mit Grott zu holen.

ja ich nutze "MAX" das heißt ich müsste es über: response = session.post("https://server.growatt.com/device/getMAXHistory", data={"typ": "MAX","sn": "abc","startDate": "2023-02-28", "endDate": "2023-02-28"}) erhalten?
Mit was muss man denn da 'start:', start, 'allDatasets:', allDatasets belegen?

ah habs sorry jetzt bin ich glücklich - danke

[MAX]: {
    snParam: 'maxSn',
    getTotalData: '/panel/max/getMAXTotalData',
    getHistory: '/device/getMAXHistory',
  },
...
getHistory(typ, sn, startDate, endDate, start, allDatasets) {
  const params = new Url.URLSearchParams({
          [GROWATTTYP[typ].snParam]: sn,
          startDate: startDate.toISOString().substring(0, 10),
          endDate: endDate.toISOString().substring(0, 10),
          start,
        });
...
      if (typeof options.historyAll === 'undefined') {
        options.historyAll = false;
      }
      if (typeof options.historyLastStartDate === 'undefined') {
        options.historyLastStartDate = new Date(new Date().setDate(new Date().getDate() - 1));
      }
      if (typeof options.historyLastEndDate === 'undefined') {
        options.historyLastEndDate = new Date(new Date().setDate(new Date().getDate() + 1));
      }
      if (typeof options.historyStart === 'undefined') {
        options.historyStart = 0;
      }
...
const historyLast = await this.getHistory(
                growattTyp,
                serialNr,
                options.historyLastStartDate,
                options.historyLastEndDate,
                options.historyStart,
                options.historyAll
              ).catch(e => {
                debugApi(`getAllPlantDeviceData getHistory ${growattTyp} err:`, e);
                reject(e);
              });

also bei MAX in json für den 1.3.2023:
{"maxSn": <xxx>,
"startDate": "2023-02-28",
"endDate": "2023-03-02",
"start": 0
}

sorry zu früh gefreut... kann das sein als ergebnis?! {'endDate': '2023-03-02', 'datas': [], 'totalPageSize': 0, 'haveNext': False}

ich glaub es liegt an dem "maxSn" oder "sn"...
was muss man da genau an Wert angeben? "maxSn": "ABC8CCU00S" oder nicht?
wobei ich jetzt DeviceType "Storage" gesehen habe als doch nicht "MAX" dann?

jetzt hab ichs wirklich - sorry für die störung

authors: Chrissili PLCHome


text:
🛑 MUSA is down

In 4fdff96, MUSA (https://storemusa.com/) was down:

HTTP code: 403
Response time: 494 ms

Resolved: MUSA is back up in 7e3b134 after 12 minutes.

repo: POCLANOS/status


text:
Footer colors and text

Make first block grey (see Figma)

@dasaderi updated on dev site to reflect FIgma https://prereview-dev.azurewebsites.net/

authors: dasaderi rudietuesdays

repo: PREreview/prereview


text:
Fixes to ensure users stay anonymous when submitting to Zenodo

Fixed the conditionals in the fullReview backend so that when a reviewer is anonymous we don't send an author object with an ORCID property to Zenodo. Part of the prior confusion on whether Zenodo lets you submit publications without an ORCID, it seems, is that Zenodo's API accepts undefined as a value for the ORCID property, but not an empty string, which gave us false signals as to what was okay.

Oops. This PR also fixes #210.


text:
build: Package prqlc as .rpm package

This uses the build-rpm-action GitHub Action to create a .rpm package for Fedora and Fedora-based Linux distributions such as AlmaLinux, CentOS and RHEL.
To install download and run:
sudo rpm -i  prqlc-0.5.3.rpm

Added a comment and merging! Thank you @vanillajonathan !

authors: max-sixty vanillajonathan

repo: PRQL/prql


text:
Missing commands in readme file

Describe the bug
The README file is missing links to the new commands Export-Resource, Export-ChildResource, Format-BicepFile and Build-BicepParamsFile

Closed in #313 and #312

authors: PalmEmanuel StefanIvemo

repo: PSBicep/PSBicep


text:
Advanced Reborn crash with Tech Reborn 1.20

Describe the bug
When only using Tech Reborn and Advanced Reborn the game crashed with reason "The game crashed whilst initializing game
Error: java.lang.RuntimeException: Could not execute entrypoint stage 'main' due to errors, provided by 'advanced_reborn'!"
Steps to reproduce the behavior:

Make a new 1.20 Fabric 0.14.21 modpack in Curse Forge Launcher
Add Tech Reborn, Advanced Reborn and Reborn Core
Start and see

Environment:
Minecraft: 1.20
Mod Loader: Fabric 0.14.21
Logs
https://pastebin.com/56FbNstR | Crash Report
https://pastebin.com/WGnBcrTx | Latest Log

thanks for your reporting.
fixed, then uploaded AR 1.2.1 and MCPitanLib 1.6.7


text:
Modify RTFM and complete example for nRF52-DK using embedded-time

Mostly, this involves modifying how RTFM interfaces with the System Timer (eg. SysTick). Currently, it's a rather "raw" interface requiring the Monotonic implementation to implement a From in which the value is converted to System Timer ticks. See rtfm-rs/cortex-m-rtfm#309.
embedded-time must first support simple comparisons and conversions not just among time units (seconds, milliseconds, etc.) (implemented transparently) but also among different underlying integer types. The latter must be done manually using the custom TryMapFrom and TryMapInto traits.

This has been completed in the following forks:
PTaylor-FluenTech/rtfm-core
PTaylor-FluenTech/cortex-m-rtfm

repo: PTaylor-FluenTech/embedded-time


text:
[Paper discussions][S1] Item-based collaborative filtering recommendation algorithms

Hi everyone!
En este issue abrimos pie a un modo de foro, en donde podrán discutir los papers de cada semana.
Vamos a ir viendo cómo va este experimento, dependiendo de su participación y la calidad de las discusiones.
Reglas:

Sólo discusiones pertinentes al paper de cada issue.
Esto no es un banco de ideas, es un lugar para resolver dudas fundamentales.
La crítica sigue siendo un ejercicio individual de reflexión y análisis.

Referencia
Sarwar, B., Karypis, G., Konstan, J., & Riedl, J. (2001). Item-based collaborative filtering recommendation algorithms. In Proceedings of the 10th international conference on World Wide Web (pp. 285-295).

Hola,
No me quedó muy claro el como encuentran similitud entre dos items, ya que hablan mucho de ratings, pero a mi parecer debiese ser algo más propio de las características del item en lugar de la calificación que se le puso, o entendí mal el concepto?

Hola,
Para el que haya entendido bien este paper, me quedó la siguiente duda:
En la sección 4.3.3 hablan de que variaron el neighborhood size para item-item, y decidieron quedarse con un óptimo de 30, pero no entiendo bien qué significa este concepto. En un principio pensaba que se referían al tamaño del set de items que se consideran similares a un item i cualquiera, sin embargo, más adelante, en la sección 4.4 se refieren a este parámetro como el model size y también lo varían, y generan los gráficos correspondientes.
Si es que hay alguien que pueda aclararme mejor este contexto, lo agradecería  :whale:

@Jiruiz2 hola, yo tampoco encuentro muy bueno el metodo de similutudes pero lo que creo entender es que si dos items tienen ratings de usuarios iguales, entonces para ambitos de recomendacion son similares.
No importa que sean dos productos completamente distinto, lo impotante es que ambos productos reciben ratings similares.
@emeiese entiendo que el model size son la cantidad de items similares que deseo almacenar en memoria y el neighbour size el numero de similitudes que ocupo para realizar una recomendacion. Es por esyo que el neighbour size debe ser igual o menor al model size ya que o si no no tendria los suficientes datos para hacer la recolendacion.

@Jiruiz2 Lo que pasa es que el paper habla de collaborative-filtering que depende de los ratings. A lo que te refieres sobre la naturaleza de los items tiene que ver con Content-based recommendation systems.
@emeiese model size k son los top k items más similares a cada item, que ocupa un espacio O(n * k) que es mucho menor que O(n*n) que sería si ocupas todos los items para precomputar similitudes. Neighbourhood size es para dado usuario u e item i, la cantidad de items de los top items similares de i que también se encuentran clasificados por el usuario u. Luego este neighborhood es usado para computar la predicción del rating.


text:
Simulated (GAZEBO) vehicle becomes unstable when trying to control a gimbal via the ActuatorControls message

The bug
Initially raised this in the sitl repo but was advised to also raise here. Simulated MAV (GAZEBO) becomes unstable when trying to control a gimbal via the ActuatorControls message (in Mavros, which in turn publishes SET_ACTUATOR_CONTROL_TARGET on mavlink). This has also been reported for LED outputs (not sure if this was also simulated or not).
To Reproduce
Steps to reproduce the behavior:

roslaunch px4 mavros_posix_sitl.launch vehicle:=typhoon_h480
Arm the vehicle and take off
Publish /mavros/actuator_control ROS message at 10hz (group_mix=2)
The UAV becomes unstable (particularly if non zero setpoints are sent to the controls field of the ActuatorControls topic).

If this process is repeated with a lower publish rate (1 hz works) the vehicle remains stable.
Drone:

Simulated typhoon h480

It seems that all offboard setpoints are ignored when the actuator setpoints are set:
https://github.com/PX4/Firmware/blob/f9324fb76a4af1e2cfe8813e1a7af5d65c2b1ed1/src/modules/mavlink/mavlink_receiver.cpp#L1000-L1011

Thanks for the report @jannsta1. I think the Typhoon H480 gimbal in SITL needs some care overall. It's not really working properly in general.

OK thanks @julianoes - if these updates are unlikely to be worked on for a while I could have a stab at them? (more than happy to leave it to the professionals if its already in the pipeline! :)). If help desired would you be able to outline any other required updates?
The only other thing I've noticed while making use of the model is that the yaw control seems a bit sluggish and/or if you fly outward in a direction from the origin and then turn around and fly back to the starting point there appears to be a large steady state error. Increasing the PID gains seems to help but I also wonder if using previous imu state for the yaw setpoint could result in some drift over time.
Also, from @Jaeyoung-Lim's comment above. it seems like the symptoms described by this issue are not actually contained in the gimbal controller gazebo plugin but are associated with the mavlink receiver. If this is the case then the repair might be a bit beyond me.

if these updates are unlikely to be worked on for a while I could have a stab at them?

I don't think anyone is currently looking into this :disappointed:, so feel free.

@jannsta1 Have your SITL simulation ever crashed when sending actuator setpoints? Sending actuator setpoints over mavros seem to make the SITL crash

@Jaeyoung-Lim - it was just the vehicle that was crashing... not gazebo itself. Although I was killing the simulation fairly soon after the loss of control.
Let me know if you want me to try anything in particular (e.g. send a particular message on a control group at a certain rate).
I wonder if you have any suggestions on fixing the central issue? assuming you have identified the root cause (which seems reasonable), a couple of ideas spring to mind:

only set the ignore_thrust / body rate... etc flags when the control group is 0, but I guess its not guaranteed that the main motors would be using control group zero.
For gimbal control it might be better to use the MAV_CMD_DO_MOUNT_CONTROL message anyway so I could try to create a new mavros plugin for this.

@jannsta1
I have tried disabling the ignore flag when the group_mlx=2 However, this still results in erratic behavior of the setpoints.
For context: I am currently sending setpoint body_rates and thrust commands that make the drone follow a trajectory, while sending actuator_control topics with group_mix=2 to control the gimbal

@Jaeyoung-Lim - could it be that another module is also publishing to the same actuator_controls topic at the same time? Having a look at the topic factor graph my interpretation is that the mavlink module would subscribe to actuator_controls_XX topics rather than publish to them?

Indeed, there is a process that is resetting the gimbal control values. I made a small fix so that at least I can control the gimbal when I stream actuator setpoints at 100Hz.
The problems that I can see is two fold.

The gimbal control topics are reset to 0 for whatever reason resulting in oscillatory behavior for low rate setpoints

The actuator outputs in SITL are as the following.

Sending actuator_setpoints(group_mix=2) at 10Hz

Sending actuator_setpoints(group_mix=2) at 100Hz

You can see that the values are reset to 0 when the actuator_setpoints (group_mix=2) are sent at a lower rate

The other setpoints: in my case the body_rate and thrust setpoints get disturbed by the actuator setpoints.

Sending actuator_setpoints(group_mix=2) at 100Hz

Sending actuator_setpoints(group_mix=2) at 100Hz

Suppressing the outputs from output_rc.cpp by commenting out the following fixes the problem.
https://github.com/PX4/Firmware/blob/5ba56c8c2cb7b48b6987ac06a503fd8841eb9925/src/modules/vmount/output_rc.cpp#L74-L84
log: https://review.px4.io/plot_app?log=b47fbcf5-babb-402e-9fdd-017e7b09c179

The pink line shows the gimbal setpoint being set continously and not being set to zero as shown in previous examples.

Thanks @Jaeyoung-Lim!
Just a quick point, I was stopping vmount (just by appending vmount stop to the file 6011_typhoon_h480.post) maybe this will prevent having to do what otherwise looks like a complicated fix?

@jannsta1 Thanks. You are right. Stopping vmount solved the problem of resetting the values to 0.

@Jaeyoung-Lim A bit off topic but have you had other problems with running the typhoon model? As of this week, if I donwload the latest repositories and try to arm/launch the typhoon_h480 model I get messages like  Preflight Fail: Compass #0 uncalibrated. All other tested models (iris_xx) seem to work fine and I've tried rolling back to different commits of sitl_gazebo and Firmware but don't seem to be able to get a working version. If needs be I will raise a separate issue for it / look into root cause but since the sitl_gazebo issues section doesn't seem to be monitored very actively it would be great to get a sanity check from someone!

@jannsta1 have you tried make clean in order to make sure the param file is deleted?

Thanks for the idea, I probably wasn't doing make clean as often as I should. Just been through the procedure of cloning the firmware again and now if I start gazebo with roslaunch px4 mavros_posix_sitl.launch vehicle:=typhoon_h480 verbose:=true I can takeoff with pxh> commander takeoff which I don't think I could do before. Also, now when I arm and send setpoints for a first time the vehicle doesn't take off. On a second attempt I get a different error - Critical navigation failure! Check sensor calibration. I guess there is something wrong with my ros node that generates setpoints but strange that this has worked fine up until now and still seems to work OK with other vehicles.

Hang on, if you're using roslaunch, you need to do this to clean the params:
rm ~/.ros/eeprom/parameters

This is a known issue and really frustrating actually, see: https://github.com/PX4/Firmware/issues/10686

Noted! But despite this I find that I can't control the typhoon_h480 model after the following Firmware commit: https://github.com/PX4/Firmware/commit/43e3fc707de823faa19b16091e5ee4a900e3c446.
The previous commit https://github.com/PX4/Firmware/commit/4a4d323a970225ec27e9cd4a409e7a287ad57ff6 and prior work fine.
I wonder if could confirm if you observe the same @julianoes or @Jaeyoung-Lim?

@jannsta Could you post this on discuss or create a separate issue for being unable to fly? It is unrelated to the issue that should be discussed here.

logged bug report https://github.com/PX4/Firmware/issues/12221 to document what I think might be a confounding issue.

@jannsta1 The MountControl plugin in mavros is now merged! You can control the gimbal continuosly while controlling the vehicle offboard.

Nice one! Will be next week before I try to integrate this but this is certainly enough to close this issue.

authors: Jaeyoung-Lim jannsta1 julianoes


text:
SITL: Master doesn't build on ubuntu 18.04

I get this:
parallels@parallels:~/repos/Firmware$ PX4_SIM_SPEED_FACTOR=2 HEADLESS=1 make px4_sitl_default gazebo_iris
[0/4] Performing build step for 'sitl_gazebo'
ninja: error: '/usr/lib/x86_64-linux-gnu/libignition-common1.so.1.0.1', needed by 'libmav_msgs.so', missing and no known rule to make it
FAILED: external/Stamp/sitl_gazebo/sitl_gazebo-build
cd /home/parallels/repos/Firmware/build/px4_sitl_default/build_gazebo && /usr/bin/cmake --build /home/parallels/repos/Firmware/build/px4_sitl_default/build_gazebo -- -j2
ninja: build stopped: subcommand failed.
Makefile:217: recipe for target 'px4_sitl_default' failed
make: *** [px4_sitl_default] Error 1

I reran the tools installs script and same thing.
The ignition on my machine has moved past 1.0.1. I am up to date on 18.04:

Can you try installing libignition-common-dev?

It's already installed:

@DonLakeFlyer I think this is something on your system, since we can build it on container environments on 18.04
Which version of gazebo are you running?

Which version of gazebo are you running?

Don't know. Whatever gets installed by the PX4 scripts. This is a clean 18.04 VM with the px4 dev toolchain installed on it.

Something broke recently I think with one of the latest 18.04 updates. Wait: I do have android studio on here as well.

Gazebo says 9.14.0

@DonLakeFlyer Okay, stange.
It looks like it simply can't find libignition-common1.so.1.0.1'

So I just updated master again. And then did a make distclean and built again and of course it works now. No idea why. I had tried that multiple time before and now suddenly it works. Anyway, sorry for the fire drill. Not sure what was happening.

@DonLakeFlyer Glad to hear that it was fixed!

authors: DonLakeFlyer Jaeyoung-Lim


text:
dcache is ok on dtcm

STM32F7 errata 1259864 Data corruption does not apply to SRAM access via the dtcm address space.
ARM will look into updating errata.
FYI: The speed slowdown is due to const data access with cache off form FLASH fetches.

I spoke to soon - there is only 128K on the DTCM. I need to speak with the STM contact.


text:
Driver framework

More initialization fixes.

Thanks!

authors: LorenzMeier erikd


text:
WIP: Manual attitude setpoint for large heading errors

mc pos control: In manual mode calculate attitude setpoint such that it reflects the users intuition of roll and pitch for any given heading error.
TODO:

[x] extensive testing in SITL
[x] testing on real multicopter
[ ] review
[x] add comments to code

Fixes #4556
Fixes #3620

A good way to test this:

Comment this line https://github.com/PX4/Firmware/blob/master/src/modules/mc_pos_control/mc_pos_control_main.cpp#L1982

Assign any value between -pi and pi to _att_sp.yaw_body here https://github.com/PX4/Firmware/blob/master/src/modules/mc_pos_control/mc_pos_control_main.cpp#L1979

Disable absolute yaw control by commenting this line https://github.com/PX4/Firmware/blob/master/src/modules/mc_att_control/mc_att_control_main.cpp#L708

By doing the above steps the copter will no longer have absolute yaw control but you can still make it turn in yaw since there is a FF on the rates. In this situation you can produce an arbitrary yaw error and verify that roll and pitch commands match the users intuition.

Log where absolute yaw control was disabled -> arbitrary high yaw error: User was still able to control roll and pitch as he expected:
http://logs.uaventure.com/view/svVXALTKZbkZbCwuyD37v8
Log where absolute yaw control was enabled
http://logs.uaventure.com/view/fBnVjFEUm9BJf6Mmhr9zhX

@tumbili The logic looks right, but I would like to clarify if the only derivation we can choose is through the proposed mix of euler angles, quaternions, trigonometric functions and rotation matrices. It would be good to have a second pass and see if we can't express this more compact using entries of a rotation matrix. I'm concerned that while we fix the common yaw case we might miss extreme angles, in particular for racers.

@LorenzMeier Tested in SITL and the behaviour is exactly the same. Maybe I can try to add even better comments.

Looks good, thanks!

Thanks for the quick fix of this issue, I'm sure a lot of users will really appreciate it. I've back ported this to master.

authors: LorenzMeier tumbili


text:
WIP: Param backend replace utarray with HashMap

Utarray uses the struct memory address as a key in a hashtable. This means we have to pass around the object instances to reference things like params and topics. This switches the backend to HashMap that does string based hashing. It will allow us to directly implement a ROS like api for param management.
I plan to keep everything 100% backwards compatible, but change the param handles to be strings internally and for new code to just uses the strings directly.
@stephanbro FYI
TODO

[x] Add HashMap submodule
[x] Move param.c to param.cpp
[ ] Create HashMap for params
[ ] Change backend to use HashMap

Useful to know: gcc does string deduplication (-fmerge-constants): https://gcc.gnu.org/onlinedocs/gcc-4.0.3/gcc/Optimize-Options.html, so 100 equal string literals will not use more Flash.
LLVM does that as well.

@LorenzMeier unclear about your comment here: https://github.com/PX4/Firmware/pull/6283#issuecomment-272396707
Do you think I'm headed in a good direction with this PR?

I just added jinja template generation for the params here. This is so I can easily swap the template now and generate the HashMap data structure.

Oh, I assumed jinja was for sitl_gazebo. Looks like I added it to the wrong image.

This now has 8 bytes/param storage size. I don't need to store the char * for the name since I'm using the dict. So (4 bytes for  float/int32 storate), 1 byte for type (uint8_t), and then 3 bytes for alignment that we could store something else there if we wanted. The existing param struct has (4 bytes for float/int32 storage), (4 bytes for const char *, and 1 bytes for type, so I think it takes up 12 bytes/param, need to check.

@dagar did I not reference docker correctly? https://github.com/PX4/Firmware/pull/6305/commits/9644f7bf9606cefde205df7ba46d94d61bde4fa0
I'm still getting issue where jinja2 is not found.

The existing param struct has (4 bytes for float/int32 storage), (4 bytes for const char *, and 1 bytes for type, so I think it takes up 12 bytes/param, need to check.

@jgoppert rebasing on master will pick up @dagar wonderful Makefile changes and we can see how the sizes change.

@jgoppert unfortunately some of it still needs to be configured on the semaphore interface. Travis-ci will need jinja installed for mac. They should all be fixed shortly.

@jgoppert should be good to go after a rebase.

@davids5 Is there somewhere I can look in the bulid log to see the size delta or do I just need to upload and check with free. Also currently this has both param systems since I haven't changed the guts of param.cpp to use the new param dictionary and am trying to do this incrementally.

@jgoppert Flash can be checked on semaphoreci - in the Build Sizes
make sizes - will also give you flash usage
For free RAM - you need to load and use free on nsh.

@dagar Once the docker image is updated on master it will need the px4tools api change for the CI: https://github.com/PX4/Firmware/pull/6305/commits/28d234af3f6c7fa2c9c8062b5fb037fb3d309f8a

The last bit of this will require some thought. I have everything working but need to decide how to handle the existing API.

Closing PR, but keeping the branch.

authors: bkueng dagar davids5 jgoppert


text:
flow terrain: use dedicated variable for last fusion time

In many places, the main EKF assumes that a recent _time_last_of_fuse means that the horizontal velocity is constrained by flow fusion. This is not the case as this variable was also used in the terrain estimator.
Also, even if in most places the flow fusion flag is checked in addition to _time_last_of_fuse, there is a time window where this condition can be true since the flag can change instantaneously.
This PR adds a new _time_last_flow_terrain_fuse to avoid confusion and bugs related to this multiple use of the same variable.

Thanks @bresch

authors: bresch dagar


text:
BMS types

I am trying to use these DSDL headers for the BMS system I'm developing. I'm having some trouble compiling them using the Nunavut CLI. I'm using the command:
nnvg --target-language c --target-endianness=little --enable-serialization-asserts c:/Path/To/Repo/public_regulated_data_types/regulated --lookup-dir c:/Path/To/Repo/public_regulated_data_types/uavcan`

After running this, I get the following error from the BatteryChemistry file.
pydsdl._data_type_builder.MissingSerializationModeError: c:\Path\To\Repo\public_regulated_data_types\regulated\drone\sensor\BatteryChemistry.1.0.uavcan:46: BatteryChemistry: Either `@sealed` or `@extent ...` are required. The smallest valid extent for this type (i.e., its max bit length) is 8 bits (1 bytes). If you are not sure what this means, add the following line near the end of this definition: `@extent 64 * 8`

After I add the line it suggests, I no longer get that error, but I have trouble with  BMSInfo file.
pydsdl._data_type_builder.AssertionCheckFailureError: c:\Users\Kyle\Documents\SkyMul\public_regulated_data_types\regulated\drone\sensor\BMSInfo.1.0.uavcan:51: Assertion check has failed

This is happening because it fails the assertion at the last line of that file.
@assert _offset_ % 8 == {0}

There is a void5 statement on the line before it, so that might be what is throwing it off.
I was able to generate the C files for the default types provided by UAVCAN, so I know I'm not doing it completely wrong. I am using Windows which I don't think has been test extensively with Nunavut.

I just tried this on Ubuntu too, and had the same issue.

I just tried this on Ubuntu too, and had the same issue.

I posted on the UAVCAN forum, and they said that this is an issue with this repository being out of date. I was able to get it to work by rebasing on the original UAVCAN repository and updating certain files with the @extent directive.

I posted on the UAVCAN forum, and they said that this is an issue with this repository being out of date. I was able to get it to work by rebasing on the original UAVCAN repository and updating certain files with the @extent directive.


text:
Chapter 1, Docker Recipe, 502 Bad Gateway

Hi
I'm new to web development and thought this book looked great to help me with Django.  But I'm stuck on the first chapter already and can't see my fault.  Please help !
After cloning the repo, I navigate to the root of the chapter_1/myproject_docker.  Before running any commands I have created this line in /etc/hosts (on my Ubuntu 18.04 OS)
127.0.0.1       myproject.local media.myproject.local static.myproject.local
And I have made sure the dev script is executable.
I then build the docker image with -
run ./bin/dev build
And finally I run -
MYSQL_USER=myproject_user MYSQL_PASSWORD=pass1234 ./bin/dev up -d
It creates the containers fine.  However when i go to "http://myproject.local" in my browser I get "502 Bad Gateway - nginx/1.17.2"
Any ideas ?

I had this. I had missed the step on added the entry into the hosts file. See page 30. You need entries for myproject.local, media.myproject.local and static.project.local
Also NGINX needs to know where to forward the request on to, and this is done through a virtual port, see page 27.
Also make sure you have set your SITE_HOST environment variable in the bin/dev.sh or bin/dev file (depending on if you used an extension)


text:
有没有完整的中文教程，能让我这种傻瓜都会使用！

我研究了两天了，还是没有搞明白，但是现在公司项目又需要自己做语音识别！
而且很急的那种········
而我又很菜，英文不好！！！
不知道有没有哪位大神有完整的操作教程！
有经验的大神笔记给抄一下也好！留个联系方式，我加你！

https://github.com/PaddlePaddle/DeepSpeech/blob/develop/README_cn.md 中文文档已经上线了 @wuzong19

repo: PaddlePaddle/DeepSpeech


text:
[CLUE] CHID 数据集可能存在评估缺失问题

https://github.com/PaddlePaddle/PaddleNLP/blob/develop/examples/benchmark/clue/mrc/run_chid.py#L529-L533
                if global_step >= num_training_steps:
                    logger.info("best_result: %.2f" % (best_acc * 100))
                    return
            tic_eval = time.time()
            acc = evaluate(model, dev_data_loader)

此处会出现，最后一个 epoch 训练完，没有评估，直接返回结果的问题。

Thanks! https://github.com/PaddlePaddle/PaddleNLP/pull/2574 will solve this bug.

authors: LiuChiachi ZHUI

repo: PaddlePaddle/PaddleNLP


text:
UIE中NER可以使用自己doccano标注的数据进行训练么？

Feature request
不做微调，直接用自己的数据进行训练，
NER中实体与bert-base有很大差异，标签就一个关键词，其他都是other，
这种我想自己训练个模型，请问下有脚本或参考文档么？
感谢！！
Motivation
因为有的很大实体识别并不仅仅限于O，per，loc，misc，time，money等等
还有可能不是这些，直接就是特定的词，不管他啥词性。
Your contribution
thx

@tonyanhq @ZeyuChen @kztao @QingshuChen
大佬请指点下啊，感谢🙏

@ucas010 Hi,
用doccano标注的NER数据可以参考UIE训练定制的流程进行训练

hi大佬，我的任务比较简单，我是用的waybill_ie，三个类别（P-B，P-I，O），但是出了个bug，
Exception in thread Thread-1:
Traceback (most recent call last):
File "/data/python3.9/threading.py", line 980, in _bootstrap_inner
self.run()
File "/data/python3.9/threading.py", line 917, in run
self._target(*self._args, **self._kwargs)
File "/data/python3.9/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 217, in _thread_loop
batch = self._dataset_fetcher.fetch(indices,
File "/data/python3.9/site-packages/paddle/fluid/dataloader/fetcher.py", line 125, in fetch
data.append(self.dataset[idx])
File "/data/python3.9/site-packages/paddlenlp/datasets/dataset.py", line 260, in getitem
return self._transform(self.new_data[idx]) if self._transform_pipline else self.new_data[idx]
File "/data/lib/python3.9/site-packages/paddlenlp/datasets/dataset.py", line 252, in _transform
data = fn(data)
File "/dataPaddleNLP/examples/information_extraction/waybill_ie/run_ernie_crf.py", line 43, in convert_to_features
tokenized_input["labels"] = [label_vocab[x] for x in labels]
File "/data/PaddleNLP/examples/information_extraction/waybill_ie/run_ernie_crf.py", line 43, in
tokenized_input["labels"] = [label_vocab[x] for x in labels]
KeyError: 'OOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOOOOOOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOP-BP-IP-IP-IOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO'

运行代码
CUDA_VISIBLE_DEVICES=1 python run_ernie_crf.py --data_dir mydata
而使用bigru_crf没有问题，但指标都是0.。。。。请问下咋办啊？
[TRAIN] Epoch:9 - Step:2769 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2770 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2771 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2772 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2773 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2774 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2775 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2776 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2777 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2778 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2779 - Loss: 0.000000
[TRAIN] Epoch:9 - Step:2780 - Loss: 0.000000
[EVAL] Precision: 0.000000 - Recall: 0.000000 - F1: 0.000000
在标记中发现，的确O比较多，所以预测的全是O，

@linjieccc @hehuang139 @sijunhe

hi大佬，我的任务比较简单，我是用的waybill_ie，三个类别（P-B，P-I，O），但是出了个bug， Exception in thread Thread-1: Traceback (most recent call last): File "/data/python3.9/threading.py", line 980, in _bootstrap_inner self.run() File "/data/python3.9/threading.py", line 917, in run self._target(*self._args, **self._kwargs) File "/data/python3.9/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 217, in _thread_loop batch = self._dataset_fetcher.fetch(indices, File "/data/python3.9/site-packages/paddle/fluid/dataloader/fetcher.py", line 125, in fetch data.append(self.dataset[idx]) File "/data/python3.9/site-packages/paddlenlp/datasets/dataset.py", line 260, in getitem return self._transform(self.new_data[idx]) if self._transform_pipline else self.new_data[idx] File "/data/lib/python3.9/site-packages/paddlenlp/datasets/dataset.py", line 252, in _transform data = fn(data) File "/dataPaddleNLP/examples/information_extraction/waybill_ie/run_ernie_crf.py", line 43, in convert_to_features tokenized_input["labels"] = [label_vocab[x] for x in labels] File "/data/PaddleNLP/examples/information_extraction/waybill_ie/run_ernie_crf.py", line 43, in tokenized_input["labels"] = [label_vocab[x] for x in labels] KeyError: 'OOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOOOOOOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOOOOOOOOOOOOOOOOP-BP-IP-IP-IP-IP-IP-IOP-BP-IP-IP-IOP-BP-IP-IP-IOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOOO'

@ucas010 方便提供一下复现代码么，我们尝试复现下这个问题

python run_ernie_crf.py --data_dir /data/PaddleNLP/examples/information_extraction/waybill_ie/data2/ --batch_size 256 --epochs 10 --save_dir /data/PaddleNLP/examples/information_extraction/waybill_ie/models/ernie_crf_ckpt/

@ucas010 方便的话可以贴一下微调用到的数据集/data/PaddleNLP/examples/information_extraction/waybill_ie/data2/


text:
[Bug]: ai studio合入微调模型killed

软件环境
- paddlepaddle-gpu: 0.0.0.post118
- paddlenlp: 2.8.0.post0

重复问题

[X] I have searched the existing issues

错误描述
硬件：ai studio V100 32G
微调模型：THUDM/chatglm2-6b

模型合并时，cpu内存占用率一直上升，直到爆内存，然后被killed。而gpu显存利用率很低，这是什么原因？
如何在合并时降低cpu的内存占用率？谢谢

稳定复现步骤 & 代码
模型微调
python finetune_generation.py chatglm2/lora_argument.json
配置文件修改
{
"dataset_name_or_path": "/home/aistudio/dataset",
"per_device_train_batch_size": 1,
"zero_padding": true,
"use_flash_attention": true,
"weight_quantize_algo": "nf4"
}
模型合并
python merge_lora_params.py
--lora_path ./checkpoints/chatglm2_lora_ckpts/checkpoint-204
--merge_lora_model_path ./checkpoints/chatglm2_lora_merge
--device "gpu"
--low_gpu_mem True
合并出错：

参数融合过程中需要将参数将在内存上进行融合，可以打开 unified checkpoint然后来避免参数融合

好的，还有一个问题，使用最新的paddlenlp3.0，相同的配置，进行模型微调时，训练一会报错，使用paddlenlp2.8不会报错：
OSError: (External) OSError: (External) CUBLAS error(14).
[Hint: 'CUBLAS_STATUS_INTERNAL_ERROR'.  An internal cuBLAS operation failed. This error is usually caused by a cudaMemcpyAsync() failure.  To correct: check that the hardware, an appropriate version of the driver, and the cuBLAS library are correctly installed. Also, check that the memory passed as a parameter to the routine is not being deallocated prior to the routine’s completion.  ] (at /paddle/paddle/phi/kernels/funcs/blas/blas_impl.cu.h:1753)

训练时显存OOM了吗？

使用ai studio自带的监视器查看，并没有，连一半都没有

参数融合过程中需要将参数将在内存上进行融合，可以打开 unified checkpoint然后来避免参数融合

好像只有训练时候才能使用，打开 unified checkpoint是不是就不用合入了

参数融合过程中需要将参数将在内存上进行融合，可以打开 unified checkpoint然后来避免参数融合

好像只有训练时候才能使用，打开 unified checkpoint是不是就不用合入了,而且paddlenlp2.8.0版本中配置文件中没有这个参数

2.8版本可以支持unified checkpoint

使用ai studio自带的监视器查看，并没有，连一半都没有

安装的paddle的cuda版本是否满足需要了？

使用ai studio自带的监视器查看，并没有，连一半都没有

安装的paddle的cuda版本是否满足需要了？
是的，使用的paddlepaddle-gpu==0.0.0.post118

参数融合过程中需要将参数将在内存上进行融合，可以打开 unified checkpoint然后来避免参数融合

好像只有训练时候才能使用，打开 unified checkpoint是不是就不用合入了,而且paddlenlp2.8.0版本中配置文件中没有这个参数

2.8版本可以支持unified checkpoint
在2.8中，微调lora命令为：python finetune_generation.py ./chatglm2/lora_argument.json,要使用unified checkpoint 需要python run_pretrain.py ./chatglm2/lora_argument.json --unified_checkpoint 1 ,这两种训练方式好像不同吧，

authors: Yang-Changhui wawltor


text:
【xft_test】add xft_test

PR types
Others
PR changes
Others
Description

 Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.You have signed the CLA already but the status is still pending? Let us recheck it.

authors: CLAassistant bukejiyu


text:
[cherry pick]disable lora

PR types
New features
PR changes
APIs
Description
cherry pick到2.8

单测和相关LLM目录没有修改

authors: lugimzzz wawltor


text:
【hydra No.17】tempoGAN

PR types

Others
PR changes

Others
Describe

https://github.com/PaddlePaddle/PaddleScience/issues/571
17 tempoGAN

PR types
Others
PR changes
Others
Describe
#571
17 tempoGAN
原代码测试运行读取数据有错误

这个bug这个pr已经改了https://github.com/PaddlePaddle/PaddleScience/pull/585，需要重新拉一下代码

这个bug如下pr已经改了： #585 需要重新拉一下代码

更新代码后可以了

repo: PaddlePaddle/PaddleScience


text:
训练4天然后突然报错了

Checklist:

查找历史相关issue寻求解答
翻阅FAQ常见问题汇总和答疑
确认bug是否在新版本里还未修复
翻阅PaddleX 使用文档

描述问题
2023-02-12 11:33:49 [INFO]      [TRAIN] Epoch=44/300, Step=1365/2425, loss_xy=4.820615, loss_wh=5.574079, loss_iou=19.460461, loss_iou_aware=4.270789, loss_obj=34.582016, loss_cls=7.113638, loss=75.821602, lr=0.000417, time_each_step=2.14s, eta=369:17:28
2023-02-12 11:34:13 [INFO]      [TRAIN] Epoch=44/300, Step=1375/2425, loss_xy=2.955108, loss_wh=3.437961, loss_iou=12.110320, loss_iou_aware=2.348799, loss_obj=20.727703, loss_cls=3.568110, loss=45.148003, lr=0.000417, time_each_step=2.38s, eta=411:55:13
2023-02-12 11:34:36 [INFO]      [TRAIN] Epoch=44/300, Step=1385/2425, loss_xy=4.763587, loss_wh=8.038626, loss_iou=21.165606, loss_iou_aware=3.268857, loss_obj=26.561720, loss_cls=4.742907, loss=68.541298, lr=0.000417, time_each_step=2.29s, eta=395:54:13
Exception in thread Thread-45:
Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 583, in _get_data
    data = self._data_queue.get(timeout=self._timeout)
  File "/root/miniconda3/lib/python3.8/multiprocessing/queues.py", line 108, in get
    raise Empty
_queue.Empty

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/root/miniconda3/lib/python3.8/threading.py", line 932, in _bootstrap_inner
    self.run()
  File "/root/miniconda3/lib/python3.8/threading.py", line 870, in run
    self._target(*self._args, **self._kwargs)
  File "/root/miniconda3/lib/python3.8/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 505, in _thread_loop
    batch = self._get_data()
  File "/root/miniconda3/lib/python3.8/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 598, in _get_data
    raise RuntimeError("DataLoader {} workers exit unexpectedly, " \
RuntimeError: DataLoader 1 workers exit unexpectedly, pids: 145393
Traceback (most recent call last):
  File "demo.py", line 46, in <module>
    model.train(
  File "/root/miniconda3/lib/python3.8/site-packages/paddlex/cv/models/detector.py", line 323, in train
    self.train_loop(
  File "/root/miniconda3/lib/python3.8/site-packages/paddlex/cv/models/base.py", line 333, in train_loop
    for step, data in enumerate(self.train_data_loader()):
  File "/root/miniconda3/lib/python3.8/site-packages/paddle/fluid/dataloader/dataloader_iter.py", line 697, in __next__
    data = self._reader.read_next_var_list()
SystemError: (Fatal) Blocking queue is killed because the data reader raises an exception.
  [Hint: Expected killed_ != true, but received killed_:1 == true:1.] (at /paddle/paddle/fluid/operators/reader/blocking_queue.h:166)

复现

您是否已经正常运行我们提供的教程？

您是否在教程的基础上修改代码内容？还请您提供运行的代码

您使用的数据集是？

请提供您出现的报错信息及相关log

环境

请提供您使用的PaddlePaddle和PaddleX的版本号

请提供您使用的操作系统信息，如Linux/Windows/MacOS

请问您使用的Python版本是？

请问您使用的CUDA/cuDNN的版本号是？

看错误是读取样本数据失败了，确认下出错时文件或者系统（如磁盘空间）是否有异常

repo: PaddlePaddle/PaddleX


text:
在paddlex在线工具箱里运行PP-YOLOE_plus_crn_s_80e_副本跑自己数据集出现api.base.utils.errors.CalledProcessError

Checklist:

查找历史相关issue寻求解答
翻阅FAQ常见问题汇总和答疑
确认bug是否在新版本里还未修复
如果bug是由PaddleX API 2.0导致，且该bug在develop分支里已修复，参考FAQ Q4替换内置PaddleX API

描述问题
在aistudio里fork了PP-YOLOE_plus_crn_s_80e，用的自己的数据集验证通过后训练报错
复现

请提供您出现的报错信息及相关log（log的查找见 FAQ Q2）

欢迎使用PaddleX！^_^
您可以在这里查看运行日志和报错信息。环境重启后运行日志将重置。
Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Traceback (most recent call last):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 209, in
main()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 205, in main
run(FLAGS, cfg)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 145, in run
trainer = Trainer(cfg, mode='train')
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/engine/trainer.py", line 101, in init
self.loader = create('{}Reader'.format(capital_mode))(
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/data/reader.py", line 168, in call
self.dataset.parse_dataset()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/data/source/coco.py", line 243, in parse_dataset
assert ct > 0, 'not found any coco record in %s' % (anno_path)
AssertionError: not found any coco record in /home/aistudio/data/ferry/annotations/instance_train.json
['/opt/conda/envs/python35-paddle120-env/bin/python', 'tools/train.py', '--eval', '--config', '/home/aistudio/.paddle_uapi/tmp8xg6_6te/detmodel_PP-YOLOE+_crn_s_80e.yml', '--use_vdl', 'True', '--vdl_log_dir', '/home/aistudio/output']
Log path: /home/aistudio/output/train.log
Traceback (most recent call last):
File "/home/aistudio/run_paddlex.py", line 52, in
runner.run()
File "/home/aistudio/base/base_run_paddlex.py", line 68, in run
self.run_train()
File "/home/aistudio/base/base_run_paddlex.py", line 267, in run_train
self.uapi_model.train(
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 84, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 86, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 97, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/runner.py", line 31, in uapi.pp_uapi.paddledet_uapi.det.runner.DetRunner.train
File "uapi/base/runner.py", line 348, in uapi.base.runner.BaseRunner.run_cmd
uapi.base.utils.errors.CalledProcessError: Command ['/opt/conda/envs/python35-paddle120-env/bin/python', 'tools/train.py', '--eval', '--config', '/home/aistudio/.paddle_uapi/tmp8xg6_6te/detmodel_PP-YOLOE+_crn_s_80e.yml', '--use_vdl', 'True', '--vdl_log_dir', '/home/aistudio/output'] returned non-zero exit status 1.
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
数据集校验成功
Warning: import ppdet from source directory without installing, run 'python setup.py install' to install ppdet firstly
loading annotations into memory...
Done (t=0.00s)
creating index...
index created!
Traceback (most recent call last):
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 209, in
main()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 205, in main
run(FLAGS, cfg)
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/tools/train.py", line 145, in run
trainer = Trainer(cfg, mode='train')
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/engine/trainer.py", line 101, in init
self.loader = create('{}Reader'.format(capital_mode))(
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/data/reader.py", line 168, in call
self.dataset.parse_dataset()
File "/opt/conda/envs/python35-paddle120-env/lib/python3.10/site-packages/uapi/pp_repos/PaddleDetection/ppdet/data/source/coco.py", line 243, in parse_dataset
assert ct > 0, 'not found any coco record in %s' % (anno_path)
AssertionError: not found any coco record in /home/aistudio/data/ferry/annotations/instance_train.json
['/opt/conda/envs/python35-paddle120-env/bin/python', 'tools/train.py', '--eval', '--config', '/home/aistudio/.paddle_uapi/tmpl1q_a_uh/detmodel_PP-YOLOE+_crn_s_80e.yml', '--use_vdl', 'True', '--vdl_log_dir', '/home/aistudio/output']
Log path: /home/aistudio/output/train.log
Traceback (most recent call last):
File "/home/aistudio/run_paddlex.py", line 52, in
runner.run()
File "/home/aistudio/base/base_run_paddlex.py", line 68, in run
self.run_train()
File "/home/aistudio/base/base_run_paddlex.py", line 267, in run_train
self.uapi_model.train(
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 84, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 86, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/model.py", line 97, in uapi.pp_uapi.paddledet_uapi.det.model.DetModel.train
File "uapi/pp_uapi/paddledet_uapi/det/runner.py", line 31, in uapi.pp_uapi.paddledet_uapi.det.runner.DetRunner.train
File "uapi/base/runner.py", line 348, in uapi.base.runner.BaseRunner.run_cmd
uapi.base.utils.errors.CalledProcessError: Command ['/opt/conda/envs/python35-paddle120-env/bin/python', 'tools/train.py', '--eval', '--config', '/home/aistudio/.paddle_uapi/tmpl1q_a_uh/detmodel_PP-YOLOE+_crn_s_80e.yml', '--use_vdl', 'True', '--vdl_log_dir', '/home/aistudio/output'] returned non-zero exit status 1.
2. 请提供您使用的GUI版本号
paddle2.5   python3.10
3. 请提供您使用的操作系统信息，如Linux/Windows/MacOS

请问您使用的CUDA/cuDNN的版本号是？

您好，请问您使用的是什么数据集呢？这个问题是必现的吗？

使用的是自定义数据集，数据集校验通过了，但是每次必现这个问题

风在动
@.***

------------------ 原始邮件 ------------------
发件人: @.>;
发送时间: 2024年4月26日(星期五) 下午3:10
收件人: @.>;
抄送: @.>; @.>;
主题: Re: [PaddlePaddle/PaddleX] 在paddlex在线工具箱里运行PP-YOLOE_plus_crn_s_80e_副本跑自己数据集出现api.base.utils.errors.CalledProcessError (Issue #1756)
您好，请问您使用的是什么数据集呢？这个问题是必现的吗？
—
Reply to this email directly, view it on GitHub, or unsubscribe.
You are receiving this because you authored the thread.Message ID: @.***>

您是否方便准备一个demo数据集呢？我们复现一下


text:
Codify short-circuit aggregation

A common pattern is to short-circuit the execution of an Aggregator if one of the upstream responses is unexpected, or if the initial request is somehow incorrect. Previously, this could have been accomplished by making the AccumulatedState an Either[HttpResponse, SomeState], and then modifying the handlers to short-circuit if the state was a Left[HttpResponse]. This quickly gets tiresome though. I thought it better to bake this pattern into the generic aggregation code, and that's what this PR does.
Along the way, I cleaned up a couple other things:

Renamed the Aggregator trait to GenericAggregator
Moved the actual aggregation logic into GenericAggregator from AggregatorController. Now the AggregatorController just calls Aggregator#execute This is good separation of concerns, and has allowed for independent tests for each GenericAggregator subclass. The tests are the same, just moved around a bit.
Made an Aggregator trait that just defines the abstract execute method. This allows for other types of aggregation (GraphQL, anyone?) and cleans up the signatures in the AggregationController considerably.

@rwalbergpd this should make your life a little easier :-).
@droustchev I mentioned this change when we chatted about aggregation today.

repo: PagerDuty/akka-http-api-gateway


text:
triggering events through EventsAPISession fails

Hi,
I'm having some issues using the EventsAPISession object. I've properly instantiated this object with a routing key obtained from  the Events API v2 integration attached to my desired service. When calling the 'trigger' method, I get a deduplicate key and even took it a step further by implementing a class that returns the full response object (see below). I can see that my calls are being returned as successful according to the response code of 202 and the response body containing {'message': 'Event processed', 'status': 'success'}.
When I visit the GUI to ensure my incident fired under the desired service, there is no instance of this incident under the service or any other service for that matter. I cannot find any breaking API changes that would be causing this, so unsure how to proceed.
from pdpyras import EventsAPISession, raise_on_error, try_decoding

class EventsAPISessionWrapper(EventsAPISession):
    def send_event(self, action, dedup_key=None, **properties):
        actions = ('trigger', 'acknowledge', 'resolve')
        if action not in actions:
            raise ValueError("Event action must be one of: "+', '.join(actions))

        event = {'event_action':action}

        event.update(properties)
        if isinstance(dedup_key, str):
            event['dedup_key'] = dedup_key
        elif not action == 'trigger':
            raise ValueError("The dedup_key property is required for"
                "event_action=%s events, and it must be a string."%action)
        response = self.post('/v2/enqueue', json=event)
        raise_on_error(response)
        response_body = try_decoding(response)
        if not 'dedup_key' in response_body:
            raise PDClientError("Malformed response body; does not contain "
                "deduplication key.", response=response)
        return response

closing, issue with PagerDuty plan, not pdpyras

repo: PagerDuty/pdpyras


text:
pkg/permissions: support negative permissions

It should be possible to declare negative permissions like so
-resource:a1:id
This would allow some sort of blacklist model like
[
   "*:*",
   "-dog:get"
]

Effectively allowing everything but getting dog

This actually causes lots of trouble in all optimizations. Not sure it'll be done

It's actually not really possible. closing for now


text:
Unsupported Transceivers

Description
In certain software versions, following an upgrade, it's observed that non-OEM Transceivers (SFPs) can become non-functional after a software upgrade. The unsupported_transceiver test fails when any such transceiver is found in output of "show system state".
Motivation and Context
If a user upgrades and is using unsupported transceivers that become unrecognized by the system, this is a potential major issue that could a severe network outage.
There are two major points up for debate before we can finish this test:

The majority of our customers may use non-OEM SFPs, which would mean this test fails all the time. As far as I can find, there is no further way to narrow down this test to a specific software version. Is this ok?
After looking at some real examples, there are optics that are listed in support documentation as "supported" but are not shown as "OEM" in the output of system state, so the test as implemented here would incorrectly mark them as failed. We could implement this list in code, but then we would be responsible for keeping it in sync which might not be good.

How Has This Been Tested?

Unit tests using real - though truncated -  show system state details

The tests have been run against vm-series firewalls but of course they don't have SFPs so we can't fully validate this without a live hardware lab.
Types of changes

New feature (non-breaking change which adds functionality)

I agree @alperenkose. Maybe we should rename the test to "non-OEM transceivers" instead?
@FoSix thoughts?

@FoSix the full list of "supported" transceivers is ONLY availalbe on the Live and requires a support login. So it's available to customers but is NOT publicly available.
I like your solution of an option to pass a list of additionally supported optics, I think that's a very clean and nice solution. I will work on it.

As discussed, added an option to provide a list of regex that can be used as a comparison while testing for OEM optics. This allows the user to specify vendor part numbers like PAN-.* so they can specify which are supported in a scalable way.

@FoSix I fixed that issue (was 3.11 prob), updated API docs and fixed the formatting issues. Anything outstanding here you think?


text:
Renaming PanObjects

I am looking to use pandevice to achieve the same effect as CLI command:
rename address TestOriginalName to TestNewName

Here's what I've tried without success:
objects.AddressObject.refreshall(Firewall, add=True)

for item in Firewall.children:
    if isinstance(item, pandevice.objects.AddressObject):
        if item.name == 'TestOriginalName':

            # This just results in completely new AddressObject being added to the firewall
            # And not a rename of the existing one like the docs for PanObject.apply() seem to imply:

            # apply()
            #   Apply this object to the device, replacing any existing object of the same name

            item.name = 'TestNewName'
            item.apply()

            # This just raises:
            # KeyError: 'name'
            item.name = 'TestNewName'
            item.update('name')

@johnworkman
Need to check and see if PAN-OS has a "rename" action, I don't know.  However, if you are looking to accomplish a rename in pandevice, you need to delete the object, then update the name, then you would .create().  We need to be slightly careful here as executing .delete() removes it from the parent's children, so we need to add it back in.  Finally, if you don't want to save the address object listing returned from the refreshall() method to loop through, you can use .find() to cut down on the code you are doing above:
from pandevice.firewall import Firewall
from pandevice.objects import AddressObject

old_name = 'TestOriginalName'
new_name = 'TestNewName'

fw = Firewall(....)
AddressObject.refreshall(fw)

target = fw.find(old_name, AddressObject)
if target is not None:
    target.delete()
    target.name = new_name
    fw.add(target)
    target.create()

authors: johnworkman shinmog

repo: PaloAltoNetworks/pandevice

license: ISC


text:
feat(module/virtual_network_gateway): New module for Site-to-Site VPN connections

Description
PR delivers new module for Site-to-Site VPN connections, which internally:

creates Azure Virtual Network Gateway
creates Azure Local Network Gateway (which in fact is customer gateway)
creates VNG connections

Motivation and Context
#303
How Has This Been Tested?
Code was tested by deploying VM-Series in common design.
In GCP cloud it was created another environment with Panorama .
Then there were established tunnels between Azure (using that module) and GCP and VM-Series from Azure were bootstrapped by Panorama in GCP.
Types of changes

New feature (non-breaking change which adds functionality)

Checklist

[x] I have updated the documentation accordingly.
[x] I have read the CONTRIBUTING document.
[ ] I have added tests to cover my changes if appropriate.
[x] All new and existing tests passed.

@FoSix according to our offline discussion, I've updated PR about:

adding azure_bgp_peers_addresses, which is a map of addresses used on Azure side (then in other parts of configuration we are referencing key)
introducing map for peering_addresses in local_bgp_settings (instead of list), in which also we are referencing to keys for IP address
merging changes from main branch e.g. removing Makefile and adding Terratest

/help

/sca


text:
切换权限用户后登录直接进404页面

Bug report（问题描述）
在一种权限用户下进入一个其独有的路由，例如 admin 进入 permission/page 页面，然后登出，再以 editor 登陆会直接 进入404
Steps to reproduce（问题复现步骤）

以admin身份登陆
从侧边栏进入 permission / Page Permission
登出当前用户
以 editor身份登入

这不是bug，因为editor没权限预览这个页面了，你可以在登出的时候，直接清空路由的redirect，让它直接跳到首页

这不是bug，因为editor没权限预览这个页面了，你可以在登出的时候，直接清空路由的redirect，让它直接跳到首页

是的，去掉redirect是解决了，但是这样使用起来确实是有问题的

repo: PanJiaChen/vue-element-admin


text:
npm ERR! D:\green\git\bin\git.EXE ls-remote -h -t ssh://git@github.com/sohee-lee7/Squire.git

Question（提问）

npm ERR! Error while executing:
npm ERR! D:\green\git\bin\git.EXE ls-remote -h -t ssh://git@github.com/sohee-lee7/Squire.git
npm ERR!
npm ERR! Host key verification failed.
npm ERR! fatal: Could not read from remote repository.
npm ERR!
npm ERR! Please make sure you have the correct access rights
npm ERR! and the repository exists.
npm ERR!
npm ERR! exited with error code: 128
想问下问题什么原因,怎么解决,我无计可施
急

我用淘宝镜像,
node.14.17.3
win10
vscode
npm config get registry
https://registry.npm.taobao.org/
翻墙能连上github

这个下载依赖失败的问题其实是因为里边配置的都是ssh  使用这个 git config --global url."https://".insteadOf ssh://git@
命令 将 ssh换成https 就能下载了，但是下载完 又给我报了个错误 。。。还在继续摸索中

这个下载依赖失败的问题其实是因为里边配置的都是ssh 使用这个 git config --global url."https://".insteadOf ssh://git@
命令 将 ssh换成https 就能下载了，但是下载完 又给我报了个错误 。。。还在继续摸索中

使用这个的前提是你能连上github。。。


text:
分页样式没有，不是引入了默认的css吗，怎么会事呢？

分页样式没有，不是引入了默认的css吗，怎么会事呢？

需要提供更详细的描述

传不了图片，我简单描述一下，麻烦了：
现在样式是这样的：
Total 50 10 /page 1234   Go to  1
正确的应该是
共 50 条  10 条/页 1234  前往 1 页

这是中英文的问题吧。。。和样式有什么关系
http://element.eleme.io/1.4/#/zh-CN/component/i18n

main.js
import locale from 'element-ui/lib/locale/lang/zh-CN'
Vue.use(ElementUI, { locale })
然后分页样式，得按照element的html结构来：

<el-pagination @size-change="handleSizeChange" @current-change="handleCurrentChange" :current-page.sync="listQuery.page"
:page-sizes="[10,20,30,50]" :page-size="listQuery.limit" layout="sizes, prev, pager, next, total" :total="total">

好的,谢谢

authors: PanJiaChen mentallxm mmd0308

repo: PanJiaChen/vueAdmin-template


text:
Describe an .abm file in the docs.

The README.md mentions an .abm file. There needs to be a description of what this file is and how it is created/obtained.

This is now described in the user guide.

repo: PandemiaProject/pandemia


text:
Adds new insert method with runtime Primary Key population

Adds a new way to recognize primary keys by introducing a new db_pk tag and new query creation and insert methods.

GenerateInsertWithPK()
InsertWithPK()

To work with primary key, there are 2 ways:

Database sequence generator (Recommended):
Whenever a primary key is marked in sql, a default constraint is typically attached to it which is responsible for populating the values for PK during insert. This is generally good when you are using auto-increment integers or uuids and don't deal with primary key during application runtime and let a sequence generator function in DB do the job. Following is how database sequence generator functions look like in sql
nextval('pura.account_account_id_seq'::regclass)

OR
uuid_generate_v4()

Here's an example of insert:
type Book struct {
    id   uint64 `db:book_id`
    name string `db:name`
}

Calling the Insert() method is the right way in those scenarios.
b := Book{"4 Hour work week"}
res := db.Schema("bookschema").Insert(b)

Application runtime generator:
Maybe, you have a scenario where you want to generate a custom value on application layer and then pass populate it as the primary key. Who knows, in those scenarios. One can add an additional tag db_pk:true to the field.
For example:
type Book struct {
    id   uint64 `db:book_id db_pk:true`
    name string `db:name`
}

then you can use the InsertWithPK() method to add your primary key values as well.
epoch := time.Now().UnixNano() / int64(time.Millisecond)
b := Book{epoch, "4 Hour work week"}
res := db.Schema("bookschema").InsertWithPK(b)

@Randy1Burrell sent the fix to capture the error, please re-review the PR.

repo: Paperchain/papergres


text:
Deprecate target:selector: asynchronous methods in all public classes.

This moves all the declarations into class extensions (categories) as well as adds deprecated attribute to all methods that end with ...target:selector:. These methods are pretty much useless and are deprecated in favor of either block-based callbacks or task-based approach.

KILL IT WITH FIRE.

authors: nlutsenko richardjrossiii

repo: ParsePlatform/Parse-SDK-iOS-OSX


text:
Failed to deploy Parse-server-example to Heroku

I cloned Parse-server-example to my GitHub as a repository, and tried to connect Heroku to the GitHub repo and deploy it to Heroku.
My package.json looks like:
{
  "name": "Parse-server-example",
  "version": "1.2.0",
  "description": "description",
  "main": "index.js",
  "repository": {
    "type": "git",
    "url": "{MY_GITHUB_REPO}"
  },
  "license": "MIT",
  "dependencies": {
    "express": "~4.11.x",
    "kerberos": "~0.0.x",
    "parse": "~1.8.0",
    "parse-server": "~2.2.2",
    "underscore": "~1.8.3",
    "cors": "*"
  },
  "scripts": {
    "start": "node index.js"
  },
  "engines": {
    "node": ">=4.3"
  }
}

The Heroku deployment console logged the following messages:
-----> Using set buildpack heroku/nodejs

-----> Node.js app detected

-----> Creating runtime environment

       NPM_CONFIG_LOGLEVEL=error

       NPM_CONFIG_PRODUCTION=true

       NODE_ENV=production

       NODE_MODULES_CACHE=true

-----> Installing binaries

       engines.node (package.json):  >=4.3

       engines.npm (package.json):   unspecified (use default)

       Resolving node version >=4.3 via semver.io...

       Downloading and installing node 5.11.1...

       Using default npm version: 3.8.6

-----> Restoring cache

       Skipping cache restore (new runtime signature)

-----> Building dependencies

       Pruning any extraneous modules

       Installing node modules (package.json)

       npm ERR! fetch failed https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.5.tgz

       npm ERR! fetch failed https://registry.npmjs.org/cookie-signature/-/cookie-signature-1.0.5.tgz

Then deployed failed.
Any idea? Thanks!

Same problem here..

After changing to "express": "4.14.0" and "parse-server": "2.2.15" I got past the cookie-signature error. But now i get the same error for
npm ERR! fetch failed https://registry.npmjs.org/jpeg-js/-/jpeg-js-0.1.2.tgz.
It seems that jpeg-js is used in "jimp": "0.2.24" that I also have in my project.json.

@oyvindvol
Thanks for that. It worked for me. Now the deployment went through.

authors: grassland-curing-cfa oyvindvol

repo: ParsePlatform/parse-server-example


text:
Cloud Code Query not being called

This query will not work in Cloud Code. I am getting the "comment is new", but nothing inside the query block. For testing I have turned off all CLPs so that shouldn't be a problem. I am running the latest version of the server locally.
Parse.Cloud.afterSave("Comments", function(request) {
  if (!request.object.existed()) {
    console.log("comment is new");
    var Posts = Parse.Object.extend("Posts");
    var query = new Parse.Query(Posts);
    console.log("post id: " + request.object.get("post").id);
    query.get(request.object.get("post").id, {
      success: function(post) {
        post.increment("commentCount");
        post.save({ useMasterKey: true });
      },
      error: function(error) {
        console.log("Got an error " + error.code + " : " + error.message);
      },
      useMasterKey: true
    });

  }

});

After a few minutes I received this:
query error ParseError {   code: 107,   message: 'Received an error with invalid JSON from Parse: <!DOCTYPE html>\n    \n    \n      <meta name="viewport" content="width=device-width, initial-scale=1">\n      \n    <title>Application Error</title>\n    \n    \n      \n
Application Error
\n      \n    \n    ' }

Any luck?

repo: ParsePlatform/parse-server


text:
Windows 8

We were using Parse.com in production and now migrated to hosted parse server.
I know as of now parse server does not support windows 8 push notifications.
Any guidance to make it work will be much appreciated.
Thanks in advance!

Hello @manishmalani where did you read this? I have been told otherwise by some MSFT reps who are catering to us to use the Microsoft Azure Parse service. I, however, don't have my app ported over and working yet. I will be very disappointed if this is the case... Windows Phone 8 seems to totally be an afterthought by Microsoft and hasn't proven to be very developer friendly at all.
Here's my article I'm referencing: https://azure.microsoft.com/en-us/blog/announcing-the-publication-of-parse-server-with-azure-managed-services/

Thanks @tetreault for your response and input.
Following wiki says only about APNS (iOS) and GCM (Android) installation on Parse server. https://github.com/ParsePlatform/parse-server/wiki/Push and under ‘Future Improvements’ topic  its mention that they will ‘support more platforms’ in future.
This is the base of my assumption that Parse server do not support windows.
We have deployed parse server on our in-house machine and organization do not intend to use paid services like Azure. For iOS and Android we got it working.

Support for Windows Phone push hasn't been added yet as far as I know. If this is something you really need and using other third party services won't fit your needs, feel free to submit a PR to add a Windows push adapter.
Closing as this is not about any specific bug.

@hramos
So the original Parse SDK did support Windows Phone 8 (since we have it working still on Windows Phone 8). But the Parse Server implementation has removed any Windows Phone 8 documentation?
This is confusing, could you help clarify this for us? We understand that we are on our own when it comes to hosting Parse, but we have been directed to use Parse Server hosted on a service like Azure but Parse Server doesn't include WP8 support but the original Parse .NET SDK does? This seems segmented and confusing. Any light you can shed on this is appreciated.

You still use the same SDKs when using Parse.com or Parse Server, they still have the same support for receiving push notifications.
Parse Server is a re-implementation of the Parse.com hosted service, meant to run on your own infra for a single Parse app, and in its initial release it did not include WP8 push notification support. There's nothing preventing anyone from adding support for this to Parse Server, however.

authors: hramos manishmalani tetreault


text:
balanced consumers not balancing nicely

First: kudos on a fine effort here. Been following along for a little while and I look forward to the next release. :beers:
I tried out the balancedconsumer this evening and while it consumes, the balancing appears not to work as I'd expect (expectations on the matter having been set by the Java high level consumer). Here's my dumb little consumer https://github.com/benauthor/storm-env/blob/master/consumers/pykafka-balanced.py. I've got a dumb little producer producing round-robin to a topic of say 6 partitions.
Expected behavior:

Start consumer A on topic & group. It consumes messages from all 6 partitions.
Start consumer B on topic & group. B gets 3 partitions, and A is rebalanced to 3.
Kill consumer B. A rebalanced again and consumes all 6 partitions.

Observed:

Start consumer A on topic & group. It consume messages from all 6 partitions.
Start consumer B on topic & group. B gets 3 partitions. A stalls, stops consuming messages.
Kill consumer B. A still stalled, consuming no messages. (Alternatively, kill consumer A. B continues to consume only 3 partitions).

Zookeeper log during startup of A and then B:
(A starts here)

[2015-05-12 05:13:24,893] INFO Accepted socket connection from /127.0.0.1:42695 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-05-12 05:13:24,893] INFO Client attempting to establish new session at /127.0.0.1:42695 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-12 05:13:24,895] INFO Established session 0x14d464a7c860023 with negotiated timeout 6000 for client /127.0.0.1:42695 (org.apache.zookeeper.server.ZooKeeperServer)

(B starts here)

[2015-05-12 05:13:45,580] INFO Accepted socket connection from /127.0.0.1:42699 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-05-12 05:13:45,580] INFO Client attempting to establish new session at /127.0.0.1:42699 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-12 05:13:45,585] INFO Established session 0x14d464a7c860024 with negotiated timeout 6000 for client /127.0.0.1:42699 (org.apache.zookeeper.server.ZooKeeperServer)
[2015-05-12 05:13:45,599] INFO Got user-level KeeperException when processing sessionid:0x14d464a7c860024 type:create cxid:0xb zxid:0x282 txntype:-1 reqpath:n/a Error Path:/consumers/mygroup/owners/test2/0-1 Error:KeeperErrorCode = NodeExists for /consumers/mygroup/owners/test2/0-1 (org.apache.zookeeper.server.PrepRequestProcessor)
[2015-05-12 05:13:45,603] INFO Got user-level KeeperException when processing sessionid:0x14d464a7c860024 type:create cxid:0xf zxid:0x283 txntype:-1 reqpath:n/a Error Path:/consumers/mygroup/owners/test2/0-1 Error:KeeperErrorCode = NodeExists for /consumers/mygroup/owners/test2/0-1 (org.apache.zookeeper.server.PrepRequestProcessor)

Nothing much of interest in kafka log.
I was on one occasion able to get two consumers both consuming their partitions simultanously by starting them in close succession. But killing one produced a similar result to that described above.

filed against current master, which is at 7a8442a63d4927a

Would you mind verifying you ran this against the most recenter master? This sounds very much like a bug I fixed yesterday.  One question, though, is the consumer at the end of the topic, waiting for more messages to be written?  It should obviously still rebalance in that case, but I think we still have some bugs to iron out for how it behaves when there are no new messages coming in.

Yes, current master:
vagrant@vagrant-ubuntu-trusty-64:~/src/pykafka$ git rev-parse HEAD
7a8442a63d4927a8f16534819f3aebf4ca480cf8

And also yes, in this scenario my producer is pretty slow and my consumers are sitting there at the end of the topic, blocking on consumer.consume(). Hope that helps!

I just tested with producer yielding a sequential infinite count to a 10-partitions topic.
The very same issue happened.
producer.py
from pykafka import KafkaClient
import time

client = KafkaClient(hosts="127.0.0.1:9092")
topic = client.topics['mytopic']
producer = topic.get_producer()
count = 0
while True:
    count += 1
    time.sleep(1)
    print('produced: %s' % (count))
    producer.produce(['msg %s' % (count)])

consumer.py
from pykafka import KafkaClient
client = KafkaClient(hosts="127.0.0.1:9092")
topic = client.topics['mytopic']
consumer = topic.get_balanced_consumer(consumer_group='group1', auto_commit_enable=True, reset_offset_on_start=True)
while True:
    message = consumer.consume()
    print(message.value)

Consumer A stalled after consumer B was started.
The following logging messages were displayed on consumer A:
2015-05-13 13:18:28,348 : INFO : Rebalancing consumer fuzzy:4f2552bf-36b1-46cc-9bc2-0d86d937551e for topic mytopic.
2015-05-13 13:18:28,360 : INFO : Balancing 2 participants for 10 partitions.
Owning 5 partitions.
My Partitions: ['mytopic-0-1', 'mytopic-0-4', 'mytopic-0-2', 'mytopic-0-0', 'mytopic-0-3']
2015-05-13 13:18:28,371 : INFO : Attempting to discover offset manager for consumer group 'group1'
2015-05-13 13:18:28,375 : INFO : Found coordinator broker with id 0
2015-05-13 13:18:28,376 : INFO : Fetching offsets
2015-05-13 13:18:28,389 : INFO : Starting 1 fetcher threads
2015-05-13 13:18:28,390 : INFO : Rebalancing Complete.
2015-05-13 13:18:28,903 : INFO : Balancing 2 participants for 10 partitions.
Owning 5 partitions.
My Partitions: ['mytopic-0-1', 'mytopic-0-4', 'mytopic-0-2', 'mytopic-0-0', 'mytopic-0-3']
2015-05-13 13:18:28,903 : INFO : Rebalancing Complete.
2015-05-13 13:18:29,793 : INFO : Balancing 2 participants for 10 partitions.
Owning 5 partitions.
My Partitions: ['mytopic-0-1', 'mytopic-0-4', 'mytopic-0-2', 'mytopic-0-0', 'mytopic-0-3']
2015-05-13 13:18:29,803 : INFO : Rebalancing Complete.
2015-05-13 13:18:30,376 : INFO : Balancing 2 participants for 10 partitions.
Owning 5 partitions.
My Partitions: ['mytopic-0-1', 'mytopic-0-4', 'mytopic-0-2', 'mytopic-0-0', 'mytopic-0-3']
2015-05-13 13:18:30,377 : INFO : Rebalancing Complete.
2015-05-13 13:18:30,623 : INFO : Balancing 2 participants for 10 partitions.
Owning 5 partitions.
My Partitions: ['mytopic-0-1', 'mytopic-0-4', 'mytopic-0-2', 'mytopic-0-0', 'mytopic-0-3']
2015-05-13 13:18:30,623 : INFO : Rebalancing Complete.
2015-05-13 13:19:28,379 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:19:28,380 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:20:28,434 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:20:28,434 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:21:28,497 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:21:28,497 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:22:28,556 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:22:28,557 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:23:28,636 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:23:28,636 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:24:28,700 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:24:28,701 : INFO : Committing offsets for 5 partitions to broker id 0
2015-05-13 13:25:28,762 : INFO : Autocommitting consumer offset for consumer group group1 and topic mytopic
2015-05-13 13:25:28,763 : INFO : Committing offsets for 5 partitions to broker id 0

I've managed to replicate this bug locally using a 10-partition cluster and the producer and consumer files that @eltermann provided, and I'm currently working on finding the root cause.

As it turns out, this is happening when this loop never exits, causing the consumer.consume() call in your example code never to return. This is due to a combination of kwarg settings: consume's block parameter defaults to True and _consumer_timeout_ms defaults to -1. This combination causes consume() to block forever.

@eltermann @benauthor @kbourgoin Please pull master and run your test cases again. I believe this commit fixes this issue. See the commit message for a detailed explanation of the error mode you discovered.
One caveat to this fix, which is mentioned in the commit message: the consumer code @eltermann provided will need to change slightly. Since this commit makes it possible for consume() to return None during a rebalance even if consumer_timeout_ms=-1 and block=True, you'll need to make sure that your consumer code can handle the result of consume() being None. For example:
from pykafka import KafkaClient
client = KafkaClient(hosts="127.0.0.1:9092")
topic = client.topics['mytopic']
consumer = topic.get_balanced_consumer(consumer_group='group1', auto_commit_enable=True, reset_offset_on_start=True)
while True:
    message = consumer.consume()
    if message is not None:  # this is new
        print(message.value)

I spoke a bit too soon - application code won't need to change, since this case is now handled by this commit.

:+1: fixes it.

party! :tada:

authors: benauthor eltermann emmett9001 kbourgoin

repo: Parsely/pykafka


text:
Remove unused walltime keyword args

Currently there is a walltime keyword arg to both the python and bash app decorators and classes-- all of which do nothing. The only thing that does anything is an undocumented reserved keyword arg walltime for bash apps. We should remove these unused keyword args.

This was implemented via proc.wait quite early on in parsl, although that implementation is, I think buggy - see issue https://github.com/Parsl/parsl/issues/2824
The python implementation is differently awkward - it cannot raise an exception in a thread exception between python VM instructions, so a long lived instruction (such as a call out to heavy C code) will block the timeout.

authors: annawoodard benclifford

repo: Parsl/parsl


text:
AppFutures do not update

AppFutures do not update when result() is called before the job has been passed to the executor.

Fixed in 0.2.0


text:
bytes strings can result in unexpected string formatting

I ran into a case with HTEX executor where address was of type bytes rather than string and this resulted in some unexpected formatting errors when formatting launch commands and such. Not sure if it matters, but using bytes was working fine when working with IPP.
Here's an example of a config:
public_ip = b'123.123.123.123'
Config(
    executors=[
        HighThroughputExecutor(
            label='htex_local',
            address=public_ip,
            cores_per_worker=1,
            max_workers=2,
            provider=AWSProvider(
                'ami-073e3f122e47832bf',
                region='us-east-2',
                key_name='mykey',
                profile="default",
                state_file='awsproviderstate.json',
                nodes_per_block=1,
                init_blocks=2,
                max_blocks=2,
                min_blocks=0,
                walltime='01:00:00',
            ),
        )
    ],
    strategy=None,
)

You can see in the logs on the server and worker that addresses were formatted incorrectly (a b is inserted)
// server parsl logs
...
2019-04-09 01:03:15.441 parsl.executors.high_throughput.executor:199 [DEBUG]  Launch command: process_worker_pool.py  --max_workers=2 -c 1 --task_url=tcp://b'123.123.123.123':54865 --result_url=tcp://b'123.123.123.123':54464 --logdir=/home/ubuntu/swag-service/runinfo/040/htex_local --hb_period=30 --hb_threshold=120
...

// worker logs
...
2019-04-09 00:56:57 parsl:480 Rank:0 [INFO]  task_url: tcp://b123.123.123.123:54579
2019-04-09 00:56:57 parsl:481 Rank:0 [INFO]  result_url: tcp://b123.123.123.123:54313
...

@macintoshpie Byte strings are quite different from regular strings, so I'm a bit puzzled as to why the IPP worked with that. Either way, for all our new executors the options are explicit about the expected types, and in the case of address options we need a string. I can't think of any actionable fix for this besides additional type checking which we generally avoid since it isn't quite pythonic.
What you are seeing in the logs is Python trying to convert a byte string to a regular string and failing at it  without realizing it.

additional type checking which we generally avoid since it isn't quite pythonic.

interested in @benclifford's take on that
duck typing is pythonic:
try:
    address.decode()
except AttributeError:
    pass

Our code base isn't particularly consistent in its handling of wrong types being passed in so I don't think we have a consistent story about who should be detecting this type mismatch.
@macintoshpie Where did the byte string come from in the real situation that this issue reflects?

I think the beginning of the story could be "the config class does some basic validation"

The byte string came from urllib. On EC2, was using instance metadata to retrieve public ip:
import urllib.request

public_ip = urllib.request.urlopen("http://169.254.169.254/latest/meta-data/public-ipv4").read()

Yeah, I think this was a more of a mistake on my end, not parsl. I just found it very confusing to have it work with IPP then when changing to HTEX trying to figure out what was wrong.
Just to confirm, I just reran it again with IPP using bytes and it worked - not sure why, its still showing the bytes string but I guess ipcontroller doesn't care for some reason (ip substituted):
2019-04-11 21:33:03.862 parsl.executors.ipp_controller:99 [DEBUG]  Starting ipcontroller with 'ipcontroller --ipython-dir=/home/ubuntu/swag-service/runinfo/080 --ip=* --profile=ec2_single_node  --location=b'123.123.123.123' '

@macintoshpie that seems a reasonable piece of code that might be expected to work or fail more gracefully.

issue #873 is a way forward for detecting this: with the work in that issue done, you should be presented with a clearer error message at the point that you attempt to initialize the high throughput executor telling you that you should have supplied str, not byte.
I will close this issue in favour of #873

Thanks for the report, @macintoshpie! This has pushed us to actually start on implementing config validation which will be a big improvement for usability 😁

authors: annawoodard benclifford macintoshpie yadudoc