text:
LevelDB Error: Too many open files

I am trying to use quickUMLS to classify a document. I try splitting the document by paragraph and then matching. It works for some documents and not in others. The main error that I get if it does not work is:
Traceback (most recent call last):
  File "cui_test.py", line 23, in <module>
    match = matcher.match(line[:int(len(line)/3)], best_match=True, ignore_syntax=False)
  File "/Users/khalid/prog/work/Call_LSTM/QuickUMLS_master/quickumls.py", line 321, in match
    matches = self._get_all_matches(ngrams)
  File "/Users/khalid/prog/work/Call_LSTM/QuickUMLS_master/quickumls.py", line 221, in _get_all_matches
    cuisem_match = sorted(self.cuisem_db.get(match))
  File "/Users/khalid/prog/work/Call_LSTM/QuickUMLS_master/toolbox.py", line 258, in get
    cuis = pickle.loads(self.cui_db.Get(db_key_encode(term)))
leveldb.LevelDBError: IO error: /Users/khalid/prog/work/quickUMLS/cui-semtypes.db/cui.leveldb/006106.ldb: Too many open files

Choosing a smaller amount of characters seems to fix the problem. This is my matcher initialization:
matcher = QuickUMLS("/Users/khalid/prog/work/quickUMLS", window=20)
And this is a sample call:
match = matcher.match(line, best_match=True, ignore_syntax=False)
I'm guessing that if there are too many cuis in one section of the text, then an error is shown. It would be a nice feature if the matcher automatically split the amount of text to match all of the cuis without error instead of having the user come up with a solution themselves.

Khalid,
Please refer to issue #20 for a fix to your problem. In short: you need to configure your OS to allow more file descriptors to be open. Feel free to re-open this issue if that doesn't work.

That would seem to fix it. Thanks for the reply.
I am posting this response as an alternative to creating larger file limits on the operating system.
I decided not to mess with kernel level settings since I am still in an early phase and the files I am scanning are relatively small. Instead, I chose to break the document into smaller portions and then match. In short, the code attempts to match the given line, but if it encounters an error then it splits the line in half and does two separate matches recursively.
def quick_match(matcher, line):
    ret = []
    try:
        tmp = matcher.match(line, best_match=True, ignore_syntax=False)
        ret.append(tmp)
    except:
        ret.append(quick_match(matcher, line[:int(len(line)/2)]))
        ret.append(quick_match(matcher, line[int(len(line)/2):]))
    return ret

authors: khalludi soldni

repo: Georgetown-IR-Lab/QuickUMLS


text:
Rename equipment resource path element

This pull request explicitly sets the exported resource name to "equipment". By default, spring-data-rest sees the equipment entity class, and generates the "equipments" resource path.

Did you try:
\@Repository("equipment")
public class EquipmentsRepository .... {

I tried it just now, and it doesn't seem to work.

Ok no worries I just thought that was it's purpose.  I'll merge what you had.

source: gharchive/pull-request

authors: lbodor oehm-smith

license_type: permissive

license_source: bigquery


text:
Update docs

Hey, is it possible to update the docs? The current ones don't mention public/private keys at all.

@orf thanks for the heads up, should be good now.

source: gharchive/issue

authors: jpadilla orf

repo: GetBlimp/django-rest-framework-jwt

license: mit


text:
TimePeriodChain not recalculating after changing an inner TimerPeriod

i think the TimePeriodChain should recalculate the duration and other properties when the child period get moved or expanded.
for example :
 TimeRange range1 = new TimeRange(DateTime.Now.Date, TimeSpan.FromDays(10));             TimeRange range2 = new TimeRange(DateTime.Now.Date.AddDays(5), TimeSpan.FromDays(2));             TimePeriodChain chain = new TimePeriodChain();             range1.ExpandEndTo(DateTime.Now.Date.AddDays(15));
after running this code the duration should be 17days but now it says 12. which is now correct.

I can help fixing it if that's ok with you

repo: Giannoudis/TimePeriodLibrary


text:
🛑 MCSkinHistory is down

In 50aec96, MCSkinHistory (https://mcskinhistory.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: MCSkinHistory is back up in 81072a6 after 8 minutes.

license: MIT

license_source: github-api


text:
Feature: Add Widget_TimeSeriesQTL() to Disp_Assess() and PD_Assess_Binary() to support QTLs

Feature Details
Add Widget_TimeSeriesQTL() so that it is created when running Study_Assess()/Make_Snapshot().

snapshot <- Make_Snapshot()

augment_snapshot <- Augment_Snapshot(
  lSnapshot = snapshot,
  cPath = system.file("snapshots", "AA-AA-000-0000", package = "gsm")
)

Widget_TimeSeriesQTL(
  qtl = "qtl0004",
  raw_results = augment_snapshot$lSnapshot$results_summary,
  raw_workflow = augment_snapshot$lSnapshot$meta_workflow,
  raw_param = augment_snapshot$lSnapshot$meta_param,
  raw_analysis = augment_snapshot$lSnapshot$results_analysis
)

Closing as incorrect implementation on my part - these should only be created in Augment_Snapshot()/wherever longitudinal data is handled.

repo: Gilead-BioStats/gsm


text:
ol worker: no two worker can run together

If one working exist, then it is impossible to run a second worker in the current machine.

Prompt the user of the syndrome
Tell the user which process is currently running. (possibly the directory it belongs to)

 ~ ol worker
using existing OL directory at /root/default-ol
2020/02/22 22:13:39 save PID 6445 to file /root/default-ol/worker/worker.pid
2020/02/22 22:13:39 Start Lambda Server
2020/02/22 22:13:39 Storage dir at /root/default-ol/worker/code
2020/02/22 22:13:39 Storage dir at /root/default-ol/worker/scratch
2020/02/22 22:13:39 Create SandboxPool
2020/02/22 22:13:39 create /sys/fs/cgroup/blkio/default-ol-sandboxes [CGROUP POOL default-ol-sandboxes]
2020/02/22 22:13:39 Cleanup Lambda Manager due to error: Mkdir /sys/fs/cgroup/blkio/default-ol-sandboxes: file exists
2020/02/22 22:13:39 remove PID file /root/default-ol/worker/worker.pid
2020/02/22 22:13:39 Mkdir /sys/fs/cgroup/blkio/default-ol-sandboxes: file exists

Resolved by adding comment in worker.

repo: GindaChen/open-lambda


text:
Gallery UI - Card Image Fix

Describe the bug
The images in the cards in Gallery are not aligned properly. Some images appear to be shifted up.

Desktop:

OS: Windows
Browser: Chrome

Additional context
With CSS change the acrds in the Gallery can be displayed as shown below:

Please assign this issue to me 😃!

@anjaliasha123 sure go ahead, Make a pr with proper screenshot

Fixed the issue


text:
something knowledgable resource

hope you get it

useful tesources


text:
Tapping “source code wrap” button permits quotes in code to be changed to curly quotes

I pasted some code into a comment and tapped the code wrap button for backticks, but the quotes all flipped.

Bug Report Dump (Auto-generated)

Version 1.22.0 (1534395229)
Device: iPhone X (iOS 12.1)
TestFlight: false

Can you give us some steps on how exactly to reproduce this?

Just paste in something like this:
<script>var test = ""</script>

Highlight it, then press the </> button. If it doesn't do it, highlight it again and do it again, or enter some text around it and try it again.

Can you check if you have Smart Punctuation turned on in Settings > General > Keyboard?

Doesn't repro for me

@rnystrom I think the quotes are already angled in your screenshots.

@BasThomas Yes, "Smart Punctuation" is enabled as it's generally the default, no? I had an issue with this on a macOS editor once but they were able to resolve it without affecting other word processors/text input except when dealing with code, and this is probably the way to go.

I haven’t changed my kb from the factory setting 🤷♀
Sent with GitHawk

I mean we can get a reference to the text, then reiterate over it and replace previously normal quotes that became angled quotes with normal ones if they are within a code block?
That sounds like a lot of effort though. 😛
Sent with GitHawk

I'm not sure but regardless it seems like smart punctuation is going to cause problems on macOS or iOS when dealing with files that are code or in markdown with embedded code.

This also happens when pasting in HTML from other issues:

Pretty sure this issue was mentioned in the past
Sent with GitHawk

One possible solution would be to have a markdown editor mode for posts
that does not do this. Unless you for sure want fancy punctuation, it’s
safer to turn off for anything that supports markdown or html.
iPhoneから送信

authors: BasThomas Huddie benguild rnystrom

repo: GitHawkApp/GitHawk


text:
Fix compile dependency.

Follow up for https://github.com/GitHubSecurityLab/CodeQL-Community-Packs/pull/71

Oops, sorry, I missed that

Oops, sorry, I missed that

No worries! 👍

authors: michaelnebel pwntester


text:
Support schedule for this 'legacy' version

I have come to use this editor as my go to when producing client documentation as it packages things up very nicely.
I've just come to re-download the dmg for a new installation and notice it's now editor-legacy.
Does this mean this editor is no longer supported? Can we see a support schedule for it? Or have all efforts gone over to the website? (Which isn't for me as client docs are private, and not publishable books)

This editor is now longer supported by we have a new editor in work, we'll probably release it in the coming months.
This new editor will be more integrated with gitbook.com but will still support all other features (export, ..).
In the mid time, you can keep using this editor, you'll get an update dialog when the new editor will be released.

authors: SamyPesse designermonkey

repo: GitbookIO/editor-legacy


text:
🛑 twoc is down

In 9078fe8, twoc (twoc.co.uk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: twoc is back up in 7c87168 after 1 hour, 12 minutes.


text:
🛑 twoc is down

In c229342, twoc (twoc.co.uk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: twoc is back up in 6a108bc after 1 hour, 16 minutes.


text:
🛑 rapishare is down

In 1816f54, rapishare (rapishare.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: rapishare is back up in da59e10 after 30 minutes.


text:
🛑 feest-start is down

In 25da629, feest-start (feest-start.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: feest-start is back up in 3c05824 after 3 hours, 42 minutes.


text:
🛑 orkut is down

In e29f6c3, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in f773136.


text:
🛑 orkut is down

In af2529c, orkut (orkut.com.br) was down:

HTTP code: 403
Response time: 244 ms

Resolved: orkut is back up in 1511c77.


text:
🛑 orkut is down

In 9e67fdf, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 6715413.


text:
🛑 orkut is down

In cef269a, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in db12c24.


text:
🛑 orkut is down

In 6c0a23b, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 723b366.


text:
🛑 hcr is down

In 784b1c0, hcr (hcr.co.uk) was down:

HTTP code: 401
Response time: 702 ms

Resolved: hcr is back up in 97e65ea.


text:
🛑 orkut is down

In 604c007, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 60167e1.


text:
🛑 ojolink is down

In c93bf74, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in b312aff.


text:
🛑 penispedia is down

In d09f5b5, penispedia (penispedia.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: penispedia is back up in d262823.


text:
🛑 hcr is down

In f150194, hcr (hcr.co.uk) was down:

HTTP code: 401
Response time: 698 ms

Resolved: hcr is back up in 6a29c9a.


text:
🛑 orkut is down

In 76c2293, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 557 ms

Resolved: orkut is back up in be56ef0.


text:
🛑 adult-mate is down

In 1f74183, adult-mate (adult-mate.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: adult-mate is back up in 534bd0e.


text:
🛑 orkut is down

In 8d4e74b, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 634 ms

Resolved: orkut is back up in 9603c80.


text:
🛑 ojolink is down

In 263099c, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 502 ms

Resolved: Ojolink is back up in 574b800.


text:
🛑 orkut is down

In 0756788, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 887 ms

Resolved: orkut is back up in 9d826de.


text:
🛑 youporn is down

In c66594c, youporn (youporn.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: youporn is back up in 2d95602.


text:
🛑 orkut is down

In 9b7cd99, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 924 ms

Resolved: orkut is back up in 1fc984e.


text:
🛑 orkut is down

In e42fcac, orkut (orkut.com.br) was down:

HTTP code: 429
Response time: 924 ms

Resolved: orkut is back up in 5a71380.


text:
🛑 ojolink is down

In 4f947d3, ojolink (ojolink.net) was down:

HTTP code: 403
Response time: 453 ms

Resolved: Ojolink is back up in d0e6e20.


text:
🛑 mynewsize is down

In 4df8da1, mynewsize (mynewsize.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: mynewsize is back up in b4debe6.


text:
🛑 orkut is down

In 283d9e6, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 11ef89e.


text:
🛑 sohu is down

In 87d330f, sohu (sohu.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: sohu is back up in d135625.


text:
🛑 wordpress is down

In 01ceb31, wordpress (wordpress.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: wordpress is back up in 55305e0.


text:
🛑 ojolink is down

In e7c4786, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in e615990.


text:
🛑 ojolink is down

In 53b7830, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in d0c5915.


text:
🛑 carlinha is down

In 18f9f25, carlinha (carlinha.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: carlinha is back up in 9fb3667.


text:
🛑 orkut is down

In 7b2a8a7, orkut (orkut.com.br) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in 5bfe03a.


text:
🛑 ojolink is down

In 23451d5, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in ae959a2 after 830 days, 21 hours, 18 minutes.


text:
🛑 ojolink is down

In acf81c1, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in af52951 after 9 minutes.


text:
🛑 carlinha is down

In 83ee8e6, carlinha (carlinha.org) was down:

HTTP code: 0
Response time: 0 ms

Resolved: carlinha is back up in 681f5fd after 18 minutes.


text:
🛑 alkon is down

In 2e402d2, alkon (alkon.com.ar) was down:

HTTP code: 0
Response time: 0 ms

Resolved: alkon is back up in f451c70 after 1 day, 13 hours, 33 minutes.


text:
🛑 hcr is down

In 909128e, hcr (hcr.co.uk) was down:

HTTP code: 429
Response time: 3004 ms

Resolved: hcr is back up in 4add9a7 after .


text:
🛑 torrentzap is down

In c32c674, torrentzap (torrentzap.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: torrentzap is back up in 995596e after 1 hour, 8 minutes.


text:
🛑 uzboys is down

In 6efa7bf, uzboys (uzboys.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: uzboys is back up in 3ac8f98 after 18 minutes.


text:
🛑 ojolink is down

In 107a129, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 154bb88 after 9 minutes.


text:
🛑 ojolink is down

In 2bd8f72, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in 8af785b after 10 minutes.


text:
🛑 ojolink is down

In 0079af5, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in c115224 after 15 minutes.


text:
🛑 orkut is down

In 975ad4c, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in d1d6248 after 7 minutes.


text:
🛑 twoc is down

In c16c887, twoc (twoc.co.uk) was down:

HTTP code: 0
Response time: 0 ms

Resolved: twoc is back up in b10c6ac after 37 minutes.


text:
🛑 ojolink is down

In 3e5eaa4, ojolink (ojolink.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ojolink is back up in bad969d after 32 minutes.


text:
🛑 rapishare is down

In 99b9659, rapishare (rapishare.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: rapishare is back up in 2975995 after 5 hours, 2 minutes.


text:
🛑 gadgethouse is down

In 4e59b42, gadgethouse (gadgethouse.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: gadgethouse is back up in e91a082 after 1 hour, 3 minutes.


text:
🛑 gadgethouse is down

In a4effce, gadgethouse (gadgethouse.nl) was down:

HTTP code: 0
Response time: 0 ms

Resolved: gadgethouse is back up in 2180f4e after 19 minutes.


text:
🛑 orkut is down

In 35aa7df, orkut (orkut.co.in) was down:

HTTP code: 0
Response time: 0 ms

Resolved: orkut is back up in ed4df62 after 8 minutes.


text:
🛑 da1234 is down

In 088db6e, da1234 (da1234.cn) was down:

HTTP code: 0
Response time: 0 ms

Resolved: da1234 is back up in a82ddb2 after 9 hours, 53 minutes.


text:
impact-graph restart on env config issues

If a variable is missed in the config (.env file) the docker image keeps restarting

@mhmdksh it will be by next changes on env configs

Removing my self from this issue as its part of the what @mhmdksh presented this morning and probably a better solution.

@MoeNick @aminlatifi I don't see how this error is related to how things are hosted or how we check things in our pipelines. To me, this seems like an error handling issue.
Scenario:
Start Command --> ENV Available? ---> YES --> App Starts Successful
                                                           |
                                                             ---> NO --> App Won't Start

However, handling ENV Errors should not be a problem when using Vault, it will help a lot tracking the missing ENVs

Can someone update me about status of this issue? is it still valid?

I added to technical debts epic, is it still valid?

@MoeNick @divine-comedian There is no enough context to what this issue is about or how it can be reproduced. Also we already have an auto-restart policy for our deployments when anything wrong happens. As well as an around the clock monitoring for logs and app health.
This is mostly done by error handling and cannot be solved by us.
Please feel free to close the issue

authors: CarlosQ96 MoeNick aminlatifi mhmdksh

repo: Giveth/impact-graph


text:
'Stenopola puncticeps cf. amazonica' raises

NoMethodError: undefined method `canonical' for #Treetop::Runtime::SyntaxNode:0x00007fa29c67bb78
... realize its no longer supported, but thought I'd log it here.

not relevant after moving to gnparser code

authors: dimus mjy

repo: GlobalNamesArchitecture/biodiversity


text:
Add relocations for internal dependencies. Closes GH-190

Add relocations for internal dependencies. Closes GH-190

Are there other dependencies which should be relocated similarly? Not libraries intended for plugin use (part of Bukkit API). Maybe asm and jline too?

I think asm and jline should be relocated as well.

Nooo, nothing should be reloacated (or only flownetworking). If plugin use custom lib, it should include that lib.

@kamcio96 but if a plugins the server to have netty 4, but it has netty 5?

so plugin has to include netty 4

Still it's an issue with custom injection, see my ticket.

I wrote, here is bug: https://github.com/GlowstonePlusPlus/Glowkit/blob/bukkit%2Bglowkit/src/main/java/org/bukkit/plugin/java/PluginClassLoader.java#L97
:stuck_out_tongue_winking_eye:

But if we relocate Netty and flow it wouldn't be an issue for anyone, the plugin needs to have it itself.

I know but this way class lookups are faster because most of the classes are in the global class loader (I think?)

It is really small difference

Okey, but even then I don't see your issue with relocating

There is no issue, ofc we can relocate it like CraftBukkit 1.7.10 (in 1.8 md_5 drop that idea and use raw package). It is only my opinion. I don't like upload plugin update's with more 2Mb.
But still, class in glowkit should be changed.

You are saying that you don't like to upload big plugin files, but also saying that if a plugin is using netty, for example, it should be shaded in the plugin. This doesn't makes sense to me, sorry. The plugin would have to include it either way it you would want to make it right. Then there wore no problems with versions.

Let's add a lib API? So plugins can request for a library to be downloaded.

It's a very good, ideas, but then there must be support for custom sources, loading the plugin main without libs from that plugin and so on. I think this would be over engineered. Also there must be support for this in spigot too, otherwise people will just keep shading

authors: Paulomart deathcap kamcio96 mastercoms

repo: GlowstonePlusPlus/GlowstonePlusPlus


text:
RFR: Bugfix linebreaks broke

One typo, and one logic error. Fixed the logic error by referring to the penn differ code.

@kevinmook this look ok?

Muy bien, gracias!

authors: aamontal kevinmook


text:
添加按钮手动点击扫描库中游戏、重新从数据源更新游戏

1.修改了游戏文件夹名称/添加新游戏后，不会自动扫描更新，只能通过添加游戏添加；建议可以添加按钮，手动点击后扫描库中游戏
2.扫描游戏/匹配数据时，可以找个位置进行展示进度
3.重新从数据源更新游戏：数据源默认VNDB，切换到其他数据源后需要每个游戏分别进行编辑数据源后更新  比较麻烦；建议可以添加按钮，手动点击后根据当前设置的信息源进行更新

目前就有这个按钮，在Library->任何游戏库->自动寻找游戏，点击这个按钮之后就会扫描库中游戏了
这个进度不太好显示，因为在扫描一个游戏的时候，99.9%的时间都在等数据库的数据回复，因此进度只能显示到目前扫到了哪个文件夹：这个进度也是可以在扫描过程中打开开游戏库看见
这个事情比较麻烦的地方是可能有些人的不同游戏就是希望不同源来解析的，那如果一键更新后可能会覆盖掉不应该覆盖的东西。我目前想法是未来在库里面多个选中游戏的功能，一键更改选中游戏的源，这样感觉会更好一些。

如果还有其他问题欢迎后续提问，这个issue我就先关了哈


text:
[FEAT] 希望能增加一个2Dfan的信息源

部分老游戏在bangumi的词条被删除了，搜索不到游戏相关信息

之前联系过2Dfan的站长了，说是有可能会影响到他们的广告数据，所以加不了2Dfan源（悲

他们有没有API?

他们有没有API?

noop

所以他们内部是如何存储数据的？

所以他们内部是如何存储数据的？

我指的是没有对外开放的api，而且也不打算给我们用

这就没办法了

有个坏主意，可以通过研究他们的APP端来获取API

有个坏主意，可以通过研究他们的APP端来获取API

方法多的是，再不济都能直接爬网页。但这个是人家明确拒接的东西，我们就别做了

2dfan不支持，没办法了


text:
nextflow.config file

Hi,
Thank you for this amazing pipeline! I would like to use geneshot, but I'm unsure how to edit the nextflow.config file to the specs of our HPC cluster (36 threads & 738 GiB RAM). Should I just put 36 and 738 each time for "cpus =" and "memory ="?
Also, when I run nextflow run Golob-Minot/geneshot --help it tells me:
nextflow.exception.AbortOperationException: Remote resource not found: https://api.github.com/repos/Golob-Minot/geneshot/contents/nextflow.config
Does that mean nextflow is somehow only looking for a nextflow.config file in the github repo, and not in my current and home directory, or is this just because there currently is none in my home or current directory?
Best,
Sam

Hi Sam,
Thanks for your interest!
Both of the questions that you have should be addressed by using the newest version (0.9) which is currently in development. This version is going to be set as the default sometime in the next few months, but in the meantime you can use it by adding the flag -r v0.9. You may need to nextflow pull Golob-Minot/geneshot -r v0.9 to update your local cache first.
For the resource allocation, the default behavior of geneshot should be more than sufficient. Each task will take less than the full amount available, and so you should see multiple tasks being queued up and executed in parallel.
Do make sure to also set up a nextflow.config in your home directory which describes your HPC's job execution setup -- see https://www.nextflow.io/docs/latest/config.html.
I'm going to close this issue, but feel free to email me directly at sminot at fredhutch dot org to talk further!
Best,
Sam

authors: slambrechts sminot

repo: Golob-Minot/geneshot


text:
[Port] Blob.

About the PR

ports blob from  https://github.com/Rxup/space-station-14
credit to VigersRay for original code and
Roudenn and Rxup for maintaining
Why / Balance

adds another antag to the mix
Technical details

Backmen have the  language system from EE. a lot of code is comented out to deal whit that.
Media

Requirements

[X] I have read and am following the Pull Request and Changelog Guidelines.
[X] I have added media to this PR or it does not require an ingame showcase.

Breaking changes

Changelog

:cl: VigersRay Roudenn Rxup ,and fishbait

add: Blob added

Summary by CodeRabbit
Release Notes

New Features

Introduced a comprehensive "Blob" game mechanic with various new entities, components, and systems for managing blob interactions, transformations, and gameplay.
Added night vision functionality for enhanced visual gameplay.
Implemented user interface components for blob-related actions, including chemical swaps and transformations.
Added new blob-related actions and events to enhance gameplay dynamics.
Enhanced NPC behavior with new blob and zombification mechanics.

Bug Fixes

Resolved various issues related to blob interactions and state management.

Documentation

Updated localization files with new strings and descriptions for blob mechanics and roles.

Style

Improved organization and clarity of code related to blob systems and components.

Tests

Added new tests to ensure the functionality of blob interactions and night vision features.

Chores

Cleaned up and refactored existing code to support new blob mechanics.

Did you actually yoink Blob from a russian server? Like translating it, fixing bugs and adding to Goob? This sure is nice, maybe would be also possible to do the same with Shadowlings and Vampires, think i saw them coded there

Also what are you working on rn?

Uuhhh it hurts to say but I was developing the blob update for quality-of-life and balance changes. It will take me so time to finish
Also a big mention here: blob IS NOT balanced for wizden weapons at all. Lasers on Backmen are 3-4 times stronger, so the blob is made to resist them.

Did you actually yoink Blob from a russian server? Like translating it, fixing bugs and adding to Goob? This sure is nice, maybe would be also possible to do the same with Shadowlings and Vampires, think i saw them coded there

idk who's shadowlings but vampires are shitcoded asf

Please add Rxup in credits. He was helping me so much while developing

Uuhhh it hurts to say but I was developing the blob update for quality-of-life and balance changes that still aren't in the main Backmen repo. It will take me some time to finish and add to the build, so get ready to port another blob PR again
Also a big mention here: blob IS NOT balanced for wizden weapons at all. Lasers on Backmen are 3-4 times stronger, so the blob is made to resist them.

ops. there was a QOL already that added downgrade. i was not aware you wherr doing more.
And that explains why the core is so tanky.

Wait are you not merging Blob just because people cant decide on the announcement?

JustSome1Nearby

idk just waiting for piras or durk for it i'm not really a merge guy :trollface:

Like it cant be merged or reviewed for merging untill the requested change is done? I think

Also why didnt anyone do the same "porting" before? There are indeed abductors, vampires, shadowlings on other servers... Why not yoink it?

Also why didnt anyone do the same "porting" before? There are indeed abductors, vampires, shadowlings on other servers... Why not yoink it?

WYCI bro

add blob to dynamic :godmode:

added to dynamic. as midround antag

When I forcepreset Blob after doing golobby no blob spawns (on dev map if that matters), and when I spawn in the ghost role version of blob and take the role, it errors out saying it can't.

still the same issue after that commit

When I forcepreset Blob after doing golobby no blob spawns (on dev map if that matters), and when I spawn in the ghost role version of blob and take the role, it errors out saying it can't.

did you try spawning the blob mouse?

well this definitely is not ready for merge if it doesn't work at all lmao

tested now whit gammode blob. host became a carrier, have 10 minutes to find a spot or get forced into blob

ghostrole mouse works. that spawns on the event.
there is still a bug whit ghostrole blob. this only affects when splitting the blob
I belive this but ony happens on debug mode. will look into it more.

fix whatever is causing this
also the comments

This is caused by placing a blob and trying ti take it over via ghost role.
Use force antag verb blob

This is caused by placing a blob and trying ti take it over via ghost role. Use force antag verb blob

ngl it should be able to become a ghostrole

This is caused by placing a blob and trying ti take it over via ghost role. Use force antag verb blob

ngl it should be able to become a ghostrole

yes im working on it, it does work its just a debug error, it works on server.

is this still being worked on?

It was supposed to be "next week"... Its the third "next week"

@jorgun is this abandoned?

@jorgun FIX MERGE CONFLICT

@jorgun FIX IT 💔

We are begging at this point i guess

@jorgun is this abandoned?

no, its just that anyting iu try to get rid of the debug error , and it is still there

@jorgun can you not just, idk, leave it like that? Opening Heretic's book CRASHED the debug server for a LONG TIME... if it doesnt affect the game itself... why not? We dont want IDEAL code, we just want game mechanics! The fact that Ling has ALL abilities via actions is HORRIBLE, but no one cares!

well, yeah, the code works just fine, there is a debug error that comes up becuse it moves the mind from the core to the blob eye. then tries to attacth the ghost mind to the core, but the mind is allready attatched. this is a debug error. but i cant seem to find a way to get rid of it.

well, yeah, the code works just fine, there is a debug error that comes up becuse it moves the mind from the core to the blob eye. then tries to attacth the ghost mind to the core, but the mind is allready attatched. this is a debug error. but i cant seem to find a way to get rid of it.

Leaving the debug error, how much of the blob is done?

well, yeah, the code works just fine, there is a debug error that comes up becuse it moves the mind from the core to the blob eye. then tries to attacth the ghost mind to the core, but the mind is allready attatched. this is a debug error. but i cant seem to find a way to get rid of it.

Leaving the debug error, how much of the blob is done?

its is a port, its all "done"
5 chemical types:  brute, brute+fire,  emp, explosive, toxin.
blob pod that turn corpses into blob zombies, blobbernaut ghostroles.
automatic spreading,
upgradable tiles to strong and then to reflective
transfer of core
spliting core to make a blob ghostrole.

Please for the love of Narsie push it then... Rimworld is gonna consume me, i need tge spessgame

its coming next week trust me bro

its coming next week trust me bro

All i want for Christmas is [to see Blob merged]

This is a Christmas miracle

repo: Goob-Station/Goob-Station


text:
Wont render

Hi.  I am following along with a tutorial (http://www.yeahbutisitflash.com/?p=5226) and when I try to render on google chrome the dev tools console shows this:
Pixi.js v2.2.3 - webGL      http://www.pixijs.com/    ♥♥♥     pixi.dev.is:224

referring to line# 224: console.log.apply(console, args);
this is my code:
function init(){
//console.log("It worked!");
stage = new PIXI.Stage(0x66FF99);
renderer = PIXI.autoDetectRenderer(
512,
384,
document.getElementById("game-canvas")
);
renderer.render(stage);
}

What is the issue you are having? You didn't post any errors and you added nothing to your stage so there isn't anything to render. What is the problem?
The log you posted it just letting you know that pixi is running and with what renderer.

Nothing wrong with the program, user error.  I didn't set up a server

authors: Lord-Ford englercj


text:
(Chore) setup envs for native build flows

[ ] setup .env .env.staging and .env.production

[ ] upgrade to latest react-native-dotenv
[ ] make sure both web+native use these files correctly
[ ] .env should contain all envs with defaults for development env
[ ] .staging and.production should contain specific overrides
[ ] only non ids/keys should be in env.
[ ] match with .toml

[ ] store keys/ids envs as github secret DEV_ENV, STAGING_ENV, PROD_ENV

[ ] in build read the secret key/value pairs into environment (source {$secret} maybe) or append to the .env file

[ ] trigger build when pushed to staging or production

[ ] read correct secret according to branch, with default to dev
[ ] option to manually trigger for master branch
[ ] deploy to appcenter with _dev, _qa suffix

@sirpy a few questions:

what's the desired user flow for manually triggering build for master?
please confirm if the following is right: master -> _dev, staging -> _qa, production -> stores
are all apps and versions going to be distributed via appcenter? (right now iOS is through appcenter and android through github artifacts)
should we run unit tests before compiling new versions? seems unnecessary given that we already have CI checks for PRs

not sure what you mean. I just dont want it to release in appcenter a dev app every code update to master
yes
yes
no, once code is pushed to QA/PROD branches it should just release without running tests

update: react-native-dotenv is not being used, so I removed it and instead updated react-native-config. desc updated to reflect this changes

authors: mauriguz sirpy

repo: GoodDollar/GoodDAPP


text:
dialogPolyfill.registerDialog and browsers with native dialog support

Shouldn't dialogPolyfill.registerDialog(element) transparently leave it to the browser when native <dialog> is supported?
It currently warns the user that the browser already offers support and that the polyfill may not work correctly, but still actually performing the polyfill bindings... ? (see https://github.com/GoogleChrome/dialog-polyfill/blob/master/dialog-polyfill.js#L153 )

It seems right to me to add return; after https://github.com/GoogleChrome/dialog-polyfill/blob/master/dialog-polyfill.js#L184. @samthor, should I PR this?

This has been open for a while. Any Progress?

The line number has since moved - what do you want to add again?
On 3 August 2015 at 21:46, Jeffery Utter notifications@github.com wrote:

This has been open for a while. Any Progress?
—
Reply to this email directly or view it on GitHub
https://github.com/GoogleChrome/dialog-polyfill/issues/21#issuecomment-127477351
.

How about return; after https://github.com/GoogleChrome/dialog-polyfill/blob/master/dialog-polyfill.js#L283 ?

authors: ThiRaBrTNK jeffutter pierlo-upitup samthor

repo: GoogleChrome/dialog-polyfill


text:
Puppeteer/chrome does not follow/show valid 302 redirects when the final URI is not known

We are using Puppeteer to verify that advertiser links are redirecting to the play store as they are supposed to. Advertisers use a combination of 302 server redirects and front-end javascript redirects (with 200 response codes), making Puppeteer a great choice to evaluate final redirect destinations. The issue is that 302 redirects are not being redirected to in Puppeteer.
Steps to reproduce

Puppeteer version: 0.13.0
Platform / OS version: Mac
URLs (if applicable): http://appclk.me/store.php?page=1

What steps will reproduce the problem?
(async () => {
        const browser = await puppeteer.launch();
        const page = await browser.newPage();
        await page.emulate(determineDevice(device)); // returns the correct device from DeviceDescriptors.js

        const finalResponse = await page.goto("http://appclk.me/store.php?page=1", {waitUntil: 'networkidle0'});

        process.stdout.write(JSON.stringify({
            success: true,
            statusCode: finalResponse.status,
            url: finalResponse.url
        }));

        process.exit(0);
})();

What is the expected result?
The example URL used, http://appclk.me/store.php?page=1 returns a 200 status code that uses meta and javascript redirects that point to http://appclk.me/store.php?page=2. This subsequent URL redirects with a 302 to a market link: market://details?id=com.kabam.marvelbattle. What SHOULD happen is one of the following:

The final URL after the network is idle should be http://appclk.me/store.php?page=2 with a status code of 302. We could use the Response headers to verify that it redirects to the market link.
OR The final URL after network is idle should be market://details?id=com.kabam.marvelbattle (though Chrome doesn't know how to handle this URL.)

What happens instead?
The final URL that is printed is http://appclk.me/store.php?page=1 with a status code of 200.
Notes
We are seeing the same behavior in Chrome: open a new tab with the Network dev tools open and visit the first link. You will see that the second redirect is never shown in the network tab. This is strange behavior as the network tab should show all requests and responses. Other browsers like FF do show the second request in the Network tab.
I'm aware that this is something with Chrome and not Puppeteer, but I'm posting here in the hopes that someone will point out a flag or some option I'm not aware of that correctly shows ALL network requests so that we can accomplish our goal of verifying redirects properly. Any suggestions are welcome.

I found the solution, which is to use page.setRequestInterception()
        const browser = await puppeteer.launch();
        const page = await browser.newPage();
        await page.emulate(determineDevice(device));
        await page.setRequestInterception(true);

        page.on('request', request => {
            console.log('GOT NEW REQUEST', request.url);
            request.continue();
        });

        page.on('response', response => {
            console.log('GOT NEW RESPONSE', response.status, response.headers);
        });

What's interesting here is that if I comment out await page.setRequestInterception(true); but still leave the request/response event listeners, the second request/response is NOT printed to the console. If setRequestInterception IS used, then THREE requests are logged, with the second correctly displaying a 302, and the third being the "market://" link. Why would requesting to intercept requests change the events that are fired? Is this a bug?

I'll close this issue as the original issue was resolved. Feel free to create another if someone thinks the above bug I mentioned is legitimate.

emulate

sometimes, the program goes to the event page.requestfailed rather than page.response, so can't handle the 302 status, do you hava any solutions ?

authors: eblanshey elainema

repo: GoogleChrome/puppeteer


text:
White space is getting added to header and footer while fullscreen PDF is printed

Steps to reproduce
Tell us about your environment:

Puppeteer version: 1.13.0
Platform / OS version: Linux / Ubuntu
URLs (if applicable): https://blissful-hopper-cf61f9.netlify.com/
Node.js version: 10.11.0

What steps will reproduce the problem?
Run the following piece of code
const puppeteer = require("puppeteer");

function sleep(ms) {
	return new Promise(resolve => {
		setTimeout(resolve, ms);
	});
}

(async () => {
	const browser = await puppeteer.launch({
		headless: true
	});

	const page = await browser.newPage();

	await page.goto("https://blissful-hopper-cf61f9.netlify.com/", {
		waitUntil: "networkidle2"
	});

	page.emulateMedia("print");

	let height = await page.evaluate(
		() => document.documentElement.offsetHeight
	);

	console.log("Height", height);

	await page.pdf({
		path: "hni.pdf",
		printBackground: true,
		height: height + "px"
	});

	await browser.close();
})();

What is the expected result?
Single page PDF is to be generated without any whitespace
What happens instead?
Multiple page PDF is generated with the required height but the content is broken into multiple pages with the same height and extra whitespace.
The PDF generated is like this
hni.pdf

It was an issue w/ Bootstrap CSS and not of Puppeteer.
Anyone else facing this issue can check w/ other websites and get a reference as to what is happening.
You can find the solution here


text:
URI malformed exception that is not handled

I have an issue that I think is related to https://github.com/GoogleChrome/puppeteer/pull/668
I load a page in a loop, creating a new page each time and saving a screenshot as a result. After a few iterations, an exception is raised and not handled because of a malformed URI.
I tried to print all the url that are requested and I see some stuff like:
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M13 3.4L8 9.1 3 3.4l-1.2 1L7.5 11h1l5.7-6.6z'/%3E%3Cpath fill='%231D1D1D' d='M13 19.4l-5 5.7-5-5.7-1.2 1L7.5 27h1l5.7-6.6z'/%3E%3Cpath fill='%23FF$
https://logx.optimizely.com/log/event
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M15.7 3.5c-.5.2-1.1.4-1.7.5.6-.4 1.1-1 1.4-1.6-.6.3-1.3.6-2 .7-.6-.6-1.4-1-2.3-1-1.7 0-3 1.3-3 3 0 .2 0 .5.1.7-2.6-.2-4.8-1$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M8.8 15V8.5H11l.3-2.5H8.8V4.4c0-.7.2-1.2 1.2-1.2h1.3V.9c-.2 0-1-.1-1.9-.1C7.5.8 6 2 6 4.2V6H4v2.5h2V15h2.8z'/%3E%3Cpath fil$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath d='M8 9L1 3v9a1 1 0 0 0 1.08 1h11.84A1 1 0 0 0 15 12V3z' fill='%2378777a'/%3E%3Cpath fill='%2378777a' d='M13.38 3H2.62L8 7.5 13.38 3z'/%3E%3Cpa$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%23DC4E41' d='M10.3 23c.1.4.1.7.1 1.1 0 3-2 5.2-5.1 5.2C2.4 29.3 0 26.9 0 24s2.4-5.3 5.3-5.3c1.4 0 2.6.5 3.6 1.4l-1.5 1.5c-.6-.5-1.3-.8-2$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath d='M4.9 5.5V14H2V5.5h2.9zM5.1 3c0 .4-.1.8-.4 1.1-.4.2-.7.3-1.2.3s-.9-.2-1.2-.4-.4-.7-.4-1c0-.4.1-.8.4-1.1s.7-.4 1.2-.4.9.1 1.1.4.5.6.5 1.1zM15 $
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M14.7 7.3c0-.4-.2-.7-.4-1-.2-.3-.6-.5-.9-.6-.1 0-.3-.1-.5-.1-.3 0-.7.1-1 .3-.1 0-.1.1-.2.2h-.1c-1-.6-2.1-.8-3.1-.9 0-.4 0-.$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M14.2 1.8c-.5-.5-1.2-.8-2-.8s-1.5.3-2 .8l-2 2c-.9.9-1 2.3-.4 3.4l-.6.6c-.5-.3-1-.4-1.4-.4-.8 0-1.5.3-2 .8l-2 2c-1.1 1.1-1.1$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='32' height='32' viewBox='0 0 32 32'%3E%3Cpath fill='%231D1D1D' d='M16 20l6.5-6.5L21 12l-5 5-5-5-1.5 1.5z'/%3E%3C/svg%3E
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M4 8l6.5 6.5L12 13 7 8l5-5-1.5-1.5z'/%3E%3Cpath fill='%231D1D1D' d='M4 24l6.5 6.5L12 29l-5-5 5-5-1.5-1.5z'/%3E%3Cpath fill=$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M12 8L5.5 1.5 4 3l5 5-5 5 1.5 1.5z'/%3E%3Cpath fill='%231D1D1D' d='M12 24l-6.5-6.5L4 19l5 5-5 5 1.5 1.5z'/%3E%3Cpath fill='$
https://platform.instagram.com/en_US/embeds.js
http://secure-au.imrworldwide.com/v60.js
http://fairfax-a.p.adnxs.com/ut/v3
http://acdn.adnxs.com/mediation/v2/mediation.js
http://acdn.adnxs.com/mediation/v2/mediation.js
http://rtax.criteo.com/delivery/rta/rta.js?netId=2187&cookieName=crtg_rta&rnd=74205018896&varName=crtg_content
https://fonts.googleapis.com/css?family=Merriweather:700%7CMontserrat
http://ads.fairfax.com.au/TechOps/uncached/ffxContainerTag.js?addiv=yes&astaddetails=yes&titanwrapper=no&titandebugging=no&iasmastheads=yes&iasallure=no&an_dssf=no&cachebuster=41074065&publisher_id=888233&creative_id=62275311&creative_size=620x164&width=620&height=164&re$
http://pixel.adsafeprotected.com/jload?anId=925294&campId=desktop%7Carticle%7C620x164%7C1&pubId=1549568&chanId=888233.2534826.10056106&placementId=3744151&pubOrder=$%7BIO_ID%7D&pubCreative=62275311&custom=sport&custom2=nrl&custom3=news
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='16' height='48' viewBox='0 0 16 48'%3E%3Cpath fill='%2378777A' d='M4 14V2l10 6-10 6z'/%3E%3Cpath fill='%231D1D1D' d='M4 30V18l10 6-10 6z'/%3E%3Cpath fill='%23FCFCFC' d='M4 46V34l10 6-10 6z'$
http://ib.adnxs.com/ut/v3
http://acdn.adnxs.com/mediation/v2/mediation.js
http://acdn.adnxs.com/mediation/v2/mediation.js
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='125 1287 450 48'%3E%3Cpath fill='%23FFF' d='M243.1 1302.1l-4.1 2.7 1.4 1.2v15.9c0 1.4-1.2 2.4-2.8 2.4h-.7v-19.8l-3.5-2.3-4.4 2.9.2.4s.8-.5 1-.6c0 0 1.6 1.5 1.6 2.9$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='32' height='97' viewBox='0 0 32 97'%3E%3Cpath fill='%23666' d='M10.5 24.7c0 .9-.3 1.7-1 2.3-.6.6-1.4 1-2.3 1s-1.6-.3-2.2-1-1-1.4-1-2.3.3-1.7 1-2.3 1.4-1 2.3-1 1.7.3 2.3 1c.6.6.9 1.4.9 2.3zm$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='80' height='45' viewBox='-59 1288 80 45'%3E%3Ccircle fill='%23999' cx='-36.7' cy='1310.3' r='22.3'/%3E%3Cpath d='M-54.8 1314.9h1.6v-4.6h2.9v-1.2h-2.9v-3.1h3.6v-1.2h-5.2v10.1zm8.4-7.3c-.7 0-$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='0 1 450 48'%3E%3Cpath d='M225.7 11.1c.6 0 1.1.4 1.4.9v-1.6l-.8.1.2-.7H225l.2.7-.8-.1V12c.2-.5.7-.9 1.3-.9zm-30 26.5zm0 0zm0 .1c0-.1 0-.1 0 0zm0 0c0-.1 0-.1 0 0zm2.$
data:image/svg+xml;charset=utf-8,%3Csvg width='262' height='32' viewBox='0 0 262 32' xmlns='http://www.w3.org/2000/svg'%3E%3Cpath d='M13.082 26.962c-1.848 0-3.735-.796-5.083-2.586v-7.243c1.116-2.23 3.272-3.183 5.158-3.183 3.466 0 5.66 3.103 5.66 6.446 0 3.145-1.848 6.566$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='450' height='48' viewBox='0 0 450 48'%3E%3Cpath d='M64.3 39.6L62.5 41c-.6 0-1.4 0-1.8-.6-.3-.6-.4-1.4-.4-2.1v-15l3.5-3.6-.8-.6-3.1 2.6-5.1-5.3-6.1 5.6-.1-16.8c0-.6.1-1.2.5-1.6.8-.7 2.3-.8 3$
data:image/svg+xml;charset=utf-8,%3Csvg xmlns='http://www.w3.org/2000/svg' width='255' height='48' viewBox='0 0 255 48'%3E%3Cpath fill='%23154A93' d='M197.6 35.9h1.7v-1.8h-1.7v1.8zm8.3-1.4c-.3.3-.8.5-1.5.5-.4 0-.7-.1-1-.2-.3-.2-.5-.4-.7-.6l-.4-.9-.1-1 .1-1.1.4-.9c.2-.3.4$
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-57e1d41464746d3238007df6.js
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-5993a03b64746d0ba501d96f.js
http://www.fairfaxstatic.com.au/dtm/7d5ea80d054b96730162d0905d59678c4d2bf30c/scripts/satellite-54fd356b6266390016940000.js

I wonder if it could be the source of the problem.
Steps to reproduce
Tell us about your environment:

Puppeteer version: ccf8fb168166ee12f42586e608d3557ff48edc27
Platform / OS version: debian stretch
URLs (if applicable): http://www.smh.com.au/rugby-league/league-news/the-year-in-discord-part-i-20140105-30bhz.html

What steps will reproduce the problem?
'use strict';

const puppeteer = require('puppeteer');

(async() => {
  process.on("unhandledRejection", (reason, p) => {
    console.error("Unhandled Rejection at: Promise", p, "reason:", reason);
    // application specific logging, throwing an error, or other logic here
    cont = false;
  });
  const args = [
    "--disable-gpu",
    "--disable-setuid-sandbox",
    "--force-device-scale-factor",
    "--ignore-certificate-errors",
    "--no-sandbox",
  ];
  const options = {
    args,
    headless: true,
    ignoreHTTPSErrors: true,
  };
  const browser = await puppeteer.launch(options);
  let cont = true;
  let n = 0
  while (cont) {
    console.log(n);
    n++
    try {
      const page = await browser.newPage();
      await page.setRequestInterceptionEnabled(true);
      page.on("request", (request) => {
        if (request.resourceType === "Image") {
          request.abort();
        } else {
          request.continue();
        }
      });
      await page.goto('http://www.smh.com.au/rugby-league/league-news/the-year-in-discord-part-i-20140105-30bhz.html');
      await page.screenshot({ path: 'example.png' });
      await page.close();
    } catch (error) {
      console.error("exception", error);
    }
  }
  await browser.close();

})();

save this file as minimal.js
run nodejs minimal.js
wait...
relaunch after some time if it doesn't crash

What is the expected result?
The program should run forever
What happens instead?
Note that the number of successful loop before to get the error is not always the same. I guess that it depends on the behavior of the website.
$ nodejs src/minimal.js
0
1
2
3
Unhandled Rejection at: Promise Promise {
  <rejected> URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestIntercepted (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:139:25)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47) } reason: URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestIntercepted (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:139:25)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47)
Unhandled Rejection at: Promise Promise {
  <rejected> URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestWillBeSent (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:181:27)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47) } reason: URIError: URI malformed
    at decodeURI (<anonymous>)
    at generateRequestHash (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:399:10)
    at NetworkManager._onRequestWillBeSent (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/NetworkManager.js:181:27)
    at emitOne (events.js:115:13)
    at Session.emit (events.js:210:7)
    at Session._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:199:12)
    at Connection._onMessage (/home/louis/Code/test/v2/test_server/node_modules/puppeteer/lib/Connection.js:98:19)
    at emitOne (events.js:115:13)
    at WebSocket.emit (events.js:210:7)
    at Receiver._receiver.onmessage (/home/louis/Code/test/v2/test_server/node_modules/ws/lib/WebSocket.js:143:47)

I tried the patch from #668 and I can't reproduce the error anymore with it. So I really think that it is required to sanitize (automatically or with a hook?) the urls that are requested from the pages that are navigated to.


text:
Uncaught SyntaxError: Cannot use import statement outside a module?

Hi I am using GA analytics.js and seem to get the error using base+polyfil implementation: Uncaught SyntaxError: Cannot use import statement outside a module
my setup is in head as per readme
    <script>
      // Inline code from `https://unpkg.com/web-vitals/dist/polyfill.js` here.
    </script>

so head has
<script>
!function(){var e,t,n,i,r={passive:!0,capture:!0},a=new Date,o=function(){i=[],t=-1,e=null,f(addEventListener)},c=function(i,r){e||(e=r,t=i,n=new Date,f(removeEventListener),u())},u=function(){if(t>=0&&t<n-a){var r={entryType:"first-input",name:e.type,target:e.target,cancelable:e.cancelable,startTime:e.timeStamp,processingStart:e.timeStamp+t};i.forEach((function(e){e(r)})),i=[]}},s=function(e){if(e.cancelable){var t=(e.timeStamp>1e12?new Date:performance.now())-e.timeStamp;"pointerdown"==e.type?function(e,t){var n=function(){c(e,t),a()},i=function(){a()},a=function(){removeEventListener("pointerup",n,r),removeEventListener("pointercancel",i,r)};addEventListener("pointerup",n,r),addEventListener("pointercancel",i,r)}(t,e):c(t,e)}},f=function(e){["mousedown","keydown","touchstart","pointerdown"].forEach((function(t){return e(t,s,r)}))},p="hidden"===document.visibilityState?0:1/0;addEventListener("visibilitychange",(function e(t){"hidden"===document.visibilityState&&(p=t.timeStamp,removeEventListener("visibilitychange",e,!0))}),!0);o(),self.webVitals={firstInputPolyfill:function(e){i.push(e),u()},resetFirstInputPolyfill:o,get firstHiddenTime(){return p}}}();
</script>

at end of body tag
    <script>
    (function() {
      var script = document.createElement('script');
      script.src = 'https://unpkg.com/web-vitals';
      script.onload = function() {
        webVitals.getCLS(console.log);
        webVitals.getFID(console.log);
        webVitals.getLCP(console.log);
	webVitals.getFCP(console.log);
	webVitals.getTTFB(console.log);
      }
      document.head.appendChild(script);
    }())
    </script>
<script>
import {getCLS, getFID, getLCP, getFCP, getTTFB} from 'web-vitals';

function sendToGoogleAnalytics({name, delta, id}) {
  ga('send', 'event', {
    eventCategory: 'Web Vitals',
    eventAction: name,
    eventLabel: id,
    eventValue: Math.round(name === 'CLS' ? delta * 1000 : delta),
    nonInteraction: true,
    transport: 'beacon',
  });
}

getCLS(sendToGoogleAnalytics);
getFID(sendToGoogleAnalytics);
getLCP(sendToGoogleAnalytics);
getFCP(sendToGoogleAnalytics);
getTTFB(sendToGoogleAnalytics);
</script>
</body>

also tried with
<script>
import {getCLS, getFID, getLCP, getFCP, getTTFB} from 'web-vitals/base';

and results in same  Uncaught SyntaxError: Cannot use import statement outside a module error

Try changing your "end of body tag" code to this:
<script>
import {getCLS, getFID, getLCP, getFCP, getTTFB} from 'https://unpkg.com/web-vitals?module';

function sendToGoogleAnalytics({name, delta, id}) {
  ga('send', 'event', {
    eventCategory: 'Web Vitals',
    eventAction: name,
    eventLabel: id,
    eventValue: Math.round(name === 'CLS' ? delta * 1000 : delta),
    nonInteraction: true,
    transport: 'beacon',
  });
}

getCLS(sendToGoogleAnalytics);
getFID(sendToGoogleAnalytics);
getLCP(sendToGoogleAnalytics);
getFCP(sendToGoogleAnalytics);
getTTFB(sendToGoogleAnalytics);
</script>
</body>

authors: centminmod philipwalton

repo: GoogleChrome/web-vitals


text:
Implement shared BrowserCompat shortcode

Fixes #33. This ports over the BrowserCompat shortcode to be used by both web.dev and d.c.c. The following changes have been made:

Tests are ported from mocha to ava
Required dependencies are added to package (11ty for cache plugin, browser compat data)
i18n functionality is ported over from web.dev and rewritten using fg (which already is a dependency from 11ty) instead of custom file walking. This also allows loading i18n from multiple sources, to be able to load translations from webdev-infra and d.c.c/web.dev in parallel
Shortcode is rewritten to be templatable: you can configure a njk template instead of mixing the HTML into your shortcode JS. This allows reusing the logic on both web.dev and d.c.c but having separate markup/styling.

It's been a while since I push myself to pick something up again, sorry it took so long. 😄 But I now feel good with where we ended up. Let me know what you think!
The prefix I choose for classes is .wdi-, happy to change. The fallback colors are taken from web.dev and also work for d.c.c. but would now be technically configurable by doing
.wdi-browser-compat {
  --wdi-text-color: pink;
}

in either the web.dev or the d.c.c repo.
Not ideal: browser icons were implemented as CSS background images before, they are now added to the markup as otherwise we would need a way to copy assets from webdev-infra to dist which seemed wrong. On the other hand that also saves some requests 🙂

Regarding icons, I think it is good that they live as markup in this repo, esp that they are not dcc or web.dev dependent.

authors: devnook matthiasrohmer

repo: GoogleChrome/webdev-infra


text:
Old version not changing to new version without hard refresh while dev window is open.

I have posted the following question on Stackoverflow with a bounty (until it expired), with no responses
https://stackoverflow.com/questions/51583964/workbox-old-version-not-changing-to-new-version
I am using workbox to cache my single page application. My webpack settings looks as follows:
new workboxPlugin({
	globDirectory: dist,
	globPatterns: ['**/*.js', '**/*.html'],
	swDest: path.join(dist, 'sw.js'),
	clientsClaim: true,
	skipWaiting: true,
	}),

For the most part it works. However sometimes some devices refuse to update. (I don't know if it effects all browsers but specifically Chrome: Version 67.0.3396.99 is effected)
One scenario:

I refresh the page a few times with ctrl - F5. But the new version does not appear
I open chrome development tools and do a ctrl - F5. Then it works
If I press normal F5 (with or without devtools open) it reverts to the previous version of the application. In addition, if I close the dev tools and press ctrl - F5 it also reverts.

Other devices upgrade flawlessly.

What you describe makes me wonder whether one of a few things is happening:

There's something going on with your web server's HTTP Cache-Control headers that's causing assets to be added to the browser cache, and Workbox believes that the requests for those assets do not need to be cache-busted.

You're on a device that is almost out of free storage, and your attempts to update previously cached entries are failing due to QuotaExceeded errors.

Could you pass along a few pieces of additional information if you're able to reproduce the issue?

A screenshot or an .har archive of the relevant HTTP traffic reported in the Network panel of Chrome DevTools.

A copy of any errors that might be logged in the JS console.

The contents of your generated precache manifest file, ideally both the before (pre-update) and after (post-update).

Thanks for the help.
I will post the required info when the issue happens again. Unfortunately it does not always happen. Will try to reproduce it asap.

I have not been able to reproduce the issue. Apologies for the delay. Will let you know as soon as it happens again.

Thanks for providing those details.
The .har archive shows that your new service worker file was downloaded, but it doesn't include the network requests for the precached assets. Did you see any errors logged in the JS console at the time?
Also, I noticed that you're using Workbox 2.1.3. We're now up to Workbox 3.6.1, and while I'm not aware of any specific bugs that were fixed in between the two releases that would explain what you're seeing, I'd recommend upgrading anyway—it will make it easier to debug this if you're on the latest codebase.

Hi @jeffposnick
Thanks for the help.
Happened again today. There are no JS console errors. Except for the upgrade is there anything else I can provide you that might help debugging.

I have upgraded to the latest version. Will post here if I have any further issues.

Closing due to lack of activity—if the original poster is still able to reproduce with the latest Workbox release, we can reopen.

authors: jeffposnick pirovorster

repo: GoogleChrome/workbox


text:
My WB journey so far

I'm integrating WB into my tech writing blog. Here's my thoughts on the first experience, so far.
I started with the code snippet on the Get Started > Gulp page. It took a bit of thinking to figure out how to integrate it with the Jekyll build process. Maybe the Get Started experience needs to go a step further and create a set of recipes for the most common tooling combinations?
Bikeshed: it bugged me quite a bit that the globDirectory flag has to start with ./ and that the staticFileGlobs flag has to escape the forward slash (yet none of the other flags do).
gulp.task('bundle-sw', () => {
   return wbBuild.generateSW({
     globDirectory: './app/',
     ...
     staticFileGlobs: ['**\/*.{html,js,css}'],
     ...

Once I got it set up, I opened up DevTools to confirm that WB was "precaching" all of my files. The Application panel showed about 50 resources cached, which seemed about right. That was a cool experience.
So I went offline, and things weren't working as expected. I eventually realized that the links throughout my page all use "pretty URLs", whereas WB cached the full file names, including the .html extension. Seems like a pretty common scenario, maybe WB should have a flag that handles this?
At this point, I realized that I probably need to my own service worker with a fetch listener that handles the pretty URLs. It was surprisingly hard to find out how to do this on the docs site. This seems like a core user-journey, I think it should have an entry on the "How To" page.  I think I need to use the injectManifest() method, but my first attempt hasn't worked yet.
Some general thoughts:
I'd avoid jargon-y terms like "precache" and "inject manifest" and just describe what it's doing in plain English. Or, if you do continue to use these terms, link to definitions.

Got the injectManifest() thing working. Turns out that I had set up my Jekyll config wrong and it was clobbering the generated service worker file.
I think the injectManifest() reference (or a How To guide on using WB alongside a custom SW) should specify that you need to import WB into your custom SW and define it before calling precache([]).
importScripts('workbox-sw.prod.v1.0.1.js');

const workboxSW = new goog.SWLib();

self.addEventListener('fetch', function (event) {
  console.log('requested: ', event.request);
});

workboxSW.precache([]);

I first wrote my custom SW like this, omitting the definition of workboxSW:
self.addEventListener('fetch', function (event) {
  console.log('requested: ', event.request);
});

workboxSW.precache([]);

I thought that workboxSW.precache([]) was just a keyword that would get replaced with all of the code that it would need to cache the files (including all of the other WB library code, in other words). I didn't realize that the SW needs to actually call the WB library at runtime.

The iFixit source declares WB using the old library name. I guess it should be this now:
importScripts('workbox-sw.prod.v1.0.1.js');
--- const workboxSW = new goog.SWLib();
+++ const workboxSW = new WorkboxSW();
self.addEventListener('fetch', function (event) {
console.log('requested: ', event.request);
});
workboxSW.precache([]);

Thanks for documenting your WB journey, @kaycebasques!

I started with the code snippet on the Get Started > Gulp page. It took a bit of thinking to figure out how to integrate it with the Jekyll build process. Maybe the Get Started experience needs to go a step further and create a set of recipes for the most common tooling combinations?

We initially focused on the three most common tooling setups - I like the idea of having more recipes for common tooling combinations however 👍  I could see this list growing to include Jekyll, Hugo, Wordpress etc.

Bikeshed: it bugged me quite a bit that the globDirectory flag has to start with ./ and that the staticFileGlobs flag has to escape the forward slash (yet none of the other flags do).

Could you file this API feedback as a separate bug and we can talk about whether we can improve the ergonomics of how it works there? 🐛

Once I got it set up, I opened up DevTools to confirm that WB was "precaching" all of my files. The Application panel showed about 50 resources cached, which seemed about right. That was a cool experience.

🎉 😎

So I went offline, and things weren't working as expected. I eventually realized that the links throughout my page all use "pretty URLs", whereas WB cached the full file names, including the .html extension. Seems like a pretty common scenario, maybe WB should have a flag that handles this?

@jeffposnick @gauntface When I've run into this in my own projects, I've added runtime routing helpers to handle the "pretty URLs" but this is probably not something that will be obvious to a new user. Is your take that handling this is more something we should ensure is documented or should we expose the flag/option proposed for handling it? (I may be missing a better way we handle this today)

I'd avoid jargon-y terms like "precache" and "inject manifest" and just describe what it's doing in plain English. Or, if you do continue to use these terms, link to definitions.

Workbox/Toolbox/Precache have historically opted to use these more jargon-y terms but if they're impeding new users from using our libraries, we should change that. I'm up for a glossary of terms or trying to stay away from using such jardon where we can.

The iFixit source declares WB using the old library name. I guess it should be this now:

@jeffposnick Not urgent, but if we get a chance could we update iFixit to use the latest version of Workbox now that we list it over on the examples page?

Yup—https://github.com/GoogleChrome/application-shell/commit/e4b595e20b9f22c62d4421ce666e982b9bde9765

Opened two issues pulling out some of the remaining doc issues. I'm hoping the CDN stuff will help with the issue of copying over workbox-sw.

authors: addyosmani gauntface jeffposnick kaycebasques


text:
move getMotionPreference() call in _init of

We should move this getMotionPreference() call  a level so we know what the user's preference is before we start. We should also make the default 'false' for the prerender. This becomes more important as we add more motion to the intro screen.

The main part of this is making sure it's false for the prerender, switching from the prerender to a motion-enabled main render doesn't look shit.

closed via #292
may revisit this with #264

authors: jakearchibald kosamari

repo: GoogleChromeLabs/proxx


text:
Expose OxiPNG interlacing flag

Is your feature request related to a problem? Please describe.
I'd like to quickly add interlacing to a PNG. Interlacing can help with perceived load times since it renders something onto the page sooner, even though the overall image becomes larger.
Describe the solution you'd like
OxiPNG has an existing CLI flag to add or remove interlacing from a PNG file. (The default option is to match the input file.) I'd like to add a new checkbox to the OxiPNG UI to set this flag.
Does other service/app have this feature?
It's exposed in Photoshop, GIMP, and Inkscape export dialogs but I'm not aware of online services that offer it
Additional context
https://github.com/shssoichiro/oxipng

It's already there?

Wow, I can't see. Thanks!

authors: NotWoods jakearchibald

repo: GoogleChromeLabs/squoosh


text:
Cannot specify different SSH port for Static VM

Even when using the flag --ssh_options -p 2222, the SSH command used by PerfKit then has two -p flags as follows:
/usr/bin/ssh -A -p 22 vagrant@127.0.0.1 .... -p 2222 hostname'

But, first flag still takes precedence.

Please assign to me.
Should we add a parameter on the static_vm JSON definition to specify an SSH port?

Assigned - thanks for reporting this.  The port flag is specified a lot of places, unfortunately.
I think allowing the user to specify additional options to ssh would be preferable to adding a port - then it's more general.

Just to be clear.
Is the intended behavior that whenever something like ssh_options='-p 2222' is specified that it should overwrite any flags of the default flags (in this case -p 22 -> -p 2222)?

Generally I don't think that -p 22 should be specified unless the user explicitly provides it as an SSH option. The current behavior (always specifying -p) prevents overriding the default port in ~/.ssh/config as well.

Actually, so I added the host to the SSH config, and specify the port there. Using it that way, the SSH commands work.
Now, but then it hits another bug.
When it does the SCP, or the RemoteCopy() method, it reuses the same ssh_options, but now SCP doesn't take a -p flag but a -P for the port, and throws this stack trace

Carlos did you have a pull request for this?  Is this issue closed?

Pull request is #201

authors: cmccoy meteorfox voellm

repo: GoogleCloudPlatform/PerfKitBenchmarker


text:
Add tox.ini

Adds support for executing tests via tox, for isolated consistent environments.
Not complete yet, the following are still required:

[x] Update pre-submit hooks.
[x] Update unit testing instructions.

To test:
$ [sudo] pip install tox
$ tox

For me, this yields the following:
GLOB sdist-make: /usr/local/google/home/connormccoy/git/PerfKitBenchmarker/setup.py
py27 recreate: /usr/local/google/home/connormccoy/git/PerfKitBenchmarker/.tox/py27
py27 installdeps: -rrequirements.txt, nose>=1.3, mock>=1.0.1, psutil==3.0.0
py27 inst: /usr/local/google/home/connormccoy/git/PerfKitBenchmarker/.tox/dist/perfkitbenchmarker-0.17.0.zip
py27 installed: colorama==0.3.3,colorlog==2.6.0,Jinja2==2.7.3,MarkupSafe==0.23,mock==1.0.1,nose==1.3.7,perfkitbenchmarker==0.17.0,psutil==3.0.0,python-gflags==2.0,wheel==0.24.0
py27 runtests: PYTHONHASHSEED='3809360237'
py27 runtests: commands[0] | nosetests /usr/local/google/home/connormccoy/git/PerfKitBenchmarker/tests
..........................................................................................................................
----------------------------------------------------------------------
Ran 122 tests in 0.451s

OK
__________________________________________________________________________ summary ___________________________________________________________________________
  py27: commands succeeded
  congratulations :)

LGTM

This ups presubmit time to ~6s for me. Too long?

Down to 2.3s locally. Rebased & squashed.

authors: cmccoy klausw


text:
deferred "Attempted RPC call without active security ticket" error

Expected Behavior
Enqueuing a deferred task
Actual Behavior
Attempting to enqueue a deferred task throws an error due to a missing security ticket.
Stack trace:
Traceback (most recent call last):
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 2073, in wsgi_app
    response = self.full_dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1518, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1516, in full_dispatch_request
    rv = self.dispatch_request()
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/flask/app.py", line 1502, in dispatch_request
    return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args)
  File "/workspace/backend/tasks_io/main.py", line 24, in test
    deferred.defer(do_the_thing)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/ext/deferred/deferred.py", line 305, in defer
    return task.add(queue, transactional=transactional)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 1292, in add
    return self.add_async(queue_name, transactional).get_result()
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 1288, in add_async
    return Queue(queue_name).add_async(self, transactional, rpc)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 2139, in add_async
    return self.__AddTasks(tasks,
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 2278, in __AddTasks
    return _MakeAsyncCall('BulkAdd',
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/taskqueue/taskqueue.py", line 488, in _MakeAsyncCall
    rpc.make_call(method, request, response, get_result_hook, None)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/apiproxy_stub_map.py", line 565, in make_call
    self.__rpc.MakeCall(self.__service, method, request, response)
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/api/apiproxy_rpc.py", line 133, in MakeCall
    self._MakeCallImpl()
  File "/layers/google.python.pip/pip/lib/python3.8/site-packages/google/appengine/runtime/default_api_stub.py", line 173, in _MakeCallImpl
    raise apiproxy_errors.RPCFailedError(
google.appengine.runtime.apiproxy_errors.RPCFailedError: Attempted RPC call without active security ticket

Steps to Reproduce the Problem

Create a base Flask app with use_deferred=True

main.py
from flask import Flask
from google.appengine.api import wrap_wsgi_app

app = Flask(__name__)
app.wsgi_app = wrap_wsgi_app(app.wsgi_app, use_legacy_context_mode=False, use_deferred=True)

def do_the_thing():
    print(f"We did the thing")

@app.route('/')
def index():
    from google.appengine.ext import deferred

    deferred.defer(do_the_thing)

app.yaml
runtime: python38
entrypoint: gunicorn -b :$PORT main:app
app_engine_apis: true

handlers:
  - url: /.*
    script: auto

Run the app

$ dev_appserver.py --runtime_python_path=$(which python3) app.yaml

Navigate to http://localhost:8080/ in a browser - observe the error

Specifications

Version: v0.3.1
Platform: Ubuntu 20.04.3 (also replicated on an upstream GAE deploy)

At the moment the use_legacy_context_mode=True option is the only mode supported for the SDK. The newer context mode is being worked on but is not yet compatible with the Taskqueue API.

I get this error locally with memcache and datastore as well, which use_legacy_context_mode=True and False and use_deferred=True and False.  If I push this up to appengine the memcache read/write succeeds

Unless you're running devappserver or otherwise stubbing out the memcache/datastore services, the calls will not succeed locally. These App Engine services are not accessible outside the App Engine environment.

@asriniva I'm running devappserver locally but these services still don't work. I'm using python38 with latest version of Django. We're in the migration process.

authors: ZachOrr asriniva firerabbit hassan-arrivy


text:
Update docker images for network module

This commit updates both the lint and integration test docker images for
the network module to versions that will support the Terraform 0.12
upgrade.

I signed it!


text:
Usage with a pgxpool (postgres)

Question
First, I'm very glad to have found this project. It's great! Really hoping it makes it to official status and part of the GCP CloudSQL docs soon.
I'm using a connection pool in my app, but didn't see any documentation for using the connector with a pool. And while I was able to get things working, seemingly without issue, I wanted to run this past you all to make sure I haven't missed any important details or unintended situations.
As far as I can tell, there's only one small change needed when using pgxpool instead of pgx:
dsn := "...connection string..."

// single config points to a 'pgx.ConnConfig' struct
config, err := pgx.ParseConfig(dsn)

// vs

// pool config points to a 'pgxpool.Config' struct
config, err := pgxpool.ParseConfig(dsn)

The pgxpool.Config struct doesn't expose the DialFunc directly, but instead through an additional pointer to a pgx.ConnConfig
So instead of
config.DialFunc = func(ctx context.Context, network string, instance string) (net.Conn, error) {
        // ... cloudsqlconn dialer...
}

The pool setup would be:
// access the inner ConnConfig field of the pgxpool.Config struct
config.ConnConfig.DialFunc = func(ctx context.Context, network string, instance string) (net.Conn, error) {
	// ...cloudsqlconn dialer...
}

Do you see this as the appropriate usage of a pgxpool with the cloud-sql-go-connector?
I'm not sure if the pgxpool.Config.BeforeConnect func would be a better place to modify the underlying ConnConfig. I don't think it would since the DialFunc isn't really dynamic; doesn't need to be re-defined/bound once the app has initialized. It appears the cloudsqlconn dialer is handling all the IAM cert/token housekeeping anyway...
(My app appears to be functioning as expected so far. Additional details below.)
Thanks for the hard work! It's a life saver
Additional Context
My full setup is as follows:

AppEngine Standard
Go 1.15+
cloudsqlconn v0.1.0
CloudSQL for Postgres 14.x
Private IP (GCP internal) to CloudSQL (via VPC connector)
Cloud IAM User as Postgres Role making the connection (same service account is also associated with the app engine service itself)
pgx v4 go module (pgxpool)

Dialer setup
(For reference) My dialer is configured as follows:
ctx := context.Background()
d, err = cloudsqlconn.NewDialer(ctx, cloudsqlconn.WithIAMAuthN())
...
// config is *pgxpool.Config
config.ConnConfig.DialFunc = func(ctx context.Context, network string, instance string) (net.Conn, error) {
	return d.Dial(ctx, "...<con string>...", cloudsqlconn.WithPrivateIP())
}
...
// pool created as:
pool, connErr := pgxpool.ConnectConfig(ctx, config)
...

@superguineapig You're configuring the DialFunc against a pool exactly how I'd suggest, by configuring the underlying ConnConfig. I'll add a note to our README about using pgxpool this way.

@enocom Awesome. Thanks for taking a look!

authors: enocom superguineapig


text:
add day unit to scheduler

Refresh intervals for dynamic tags are currently specified in hours. Add option to let user to choose between hours and days for scheduling dynamic tag refreshes.

Fixed.


text:
Make debian builds repeatable

This is something I've been poking around at lately. I'm opening this issue to track progress.

Pieces:
Get the output of 'debootstrap jessie $DIR' to be byte for byte repeatable.
This will probably involve storing pinned apt sources and reusing them between invocations. It will also involve stripping timestamps, etc.
Make the tarball itself repeatable
This will also involve stripping timestamps. Something like
GZIP="-n" tar --numeric-owner ...

should work.

Output of diffing two mkdebootstrap output directories:
diff: /tmp/jessie3/usr/share/vim/vimrc: Not a directory
diff: /tmp/jessie4/usr/share/vim/vimrc: Not a directory
diff: /tmp/jessie3/usr/share/vim/vimrc.tiny: Not a directory
diff: /tmp/jessie4/usr/share/vim/vimrc.tiny: Not a directory
Files /tmp/jessie3/var/cache/ldconfig/aux-cache and /tmp/jessie4/var/cache/ldconfig/aux-cache differ
Files /tmp/jessie3/var/cache/man/da/index.db and /tmp/jessie4/var/cache/man/da/index.db differ
Files /tmp/jessie3/var/cache/man/de/index.db and /tmp/jessie4/var/cache/man/de/index.db differ
Files /tmp/jessie3/var/cache/man/es/index.db and /tmp/jessie4/var/cache/man/es/index.db differ
Files /tmp/jessie3/var/cache/man/fr/index.db and /tmp/jessie4/var/cache/man/fr/index.db differ
Files /tmp/jessie3/var/cache/man/index.db and /tmp/jessie4/var/cache/man/index.db differ
Files /tmp/jessie3/var/cache/man/it/index.db and /tmp/jessie4/var/cache/man/it/index.db differ
Files /tmp/jessie3/var/cache/man/ja/index.db and /tmp/jessie4/var/cache/man/ja/index.db differ
Files /tmp/jessie3/var/cache/man/nl/index.db and /tmp/jessie4/var/cache/man/nl/index.db differ
Files /tmp/jessie3/var/cache/man/pl/index.db and /tmp/jessie4/var/cache/man/pl/index.db differ
Files /tmp/jessie3/var/cache/man/pt_BR/index.db and /tmp/jessie4/var/cache/man/pt_BR/index.db differ
Files /tmp/jessie3/var/cache/man/ru/index.db and /tmp/jessie4/var/cache/man/ru/index.db differ
Files /tmp/jessie3/var/cache/man/sv/index.db and /tmp/jessie4/var/cache/man/sv/index.db differ
Files /tmp/jessie3/var/log/alternatives.log and /tmp/jessie4/var/log/alternatives.log differ
Files /tmp/jessie3/var/log/bootstrap.log and /tmp/jessie4/var/log/bootstrap.log differ
Files /tmp/jessie3/var/log/dpkg.log and /tmp/jessie4/var/log/dpkg.log differ

Everything in /var/cache should be safe to delete. The diffs in /var/log are mainly timestamp based:
diff /tmp/jessie3/var/log/bootstrap.log /tmp/jessie4/var/log/bootstrap.log
890,891c890,891
< Local time is now:      Fri Jul 21 16:56:36 UTC 2017.
< Universal Time is now:  Fri Jul 21 16:56:36 UTC 2017.
---
> Local time is now:      Fri Jul 21 16:58:58 UTC 2017.
> Universal Time is now:  Fri Jul 21 16:58:58 UTC 2017.

Some reading: https://wiki.debian.org/ReproducibleInstalls

Some more reading: https://github.com/andsens/bootstrap-vz/tree/96086329e629f6edb83e24cee79e265e029a7f55/bootstrapvz/plugins/minimize_size

repo: GoogleCloudPlatform/debian-docker


text:
Create some docs about Forseti logs

We have some documentation on logs and an FAQ on how to see the Forseti output logs, but we should make a separate page for it (easier to find).
Notably, we should at least mention that the GCE instance logs for the Forseti instance can be filtered on the text "[forseti-security]".

it's here: http://forsetisecurity.org/docs/howto/deploy/gcp-deployment.html#viewing-forseti-logs

repo: GoogleCloudPlatform/forseti-security


text:
Add full stack example with Flutter frontend and Cloud Run backend

Very simple demo that shows a Flutter app using a Dart functions app for a greeting API. The README docs detail building and testing locally and then deploying to Cloud Run.
This pull requests also includes community contributions that were made in here. Thanks!

What can we do to get CI validating this?

authors: kevmoo subfuzion


text:
bigquery: allow creation of date-partitioned tables

As far as I can tell, it's not possible to create a date-partitioned table via https://godoc.org/cloud.google.com/go/bigquery#Table.Create currently. This would be wonderful to be able to do (not sure how inserting into them works right now, either)

@fortytw2 I think this lets you create date partitioned tables: https://github.com/GoogleCloudPlatform/google-cloud-go/blob/master/bigquery/table.go#L183
For example:
	ctx := context.Background()
	client, err := bigquery.NewClient(ctx, "project-id")
	if err != nil {
               return err
	}

	table := client.Dataset("my_dataset").Table("new-table")
	if err := table.Create(ctx, bigquery.TimePartitioning{}); err != nil {
               return err
        }

Weird! I definitely missed that, huge thanks @BlueMonday !

repo: GoogleCloudPlatform/google-cloud-go


text:
Error, client state: -1

in the esp32-mqtt example I obtain always this result. I'm using platformio with platform espressif 32 v1.5.0.

Your buffer size is too short

the same problem with 2048 also! :(

1024 2048 for MQTT_MAX_PACKET_SIZE produce the same error (also directly into the CloudIoTCoreMQTTClient.h file and print the value on serial in order to verify that it's ok) .

This could also be caused by incorrect configuration. Maybe try eliminating one of the variables and use one of the HTTP examples; if it doesn't work with HTTP, it definitely won't work with the MQTT. Also, if you're using the library from before 1.0.4, there was a bug where the JWT was not correctly getting generated, try updating the library too.

I used the last versione (1.0.5) for my test with 1024 and 2048 buffer size. I'm working on ESP32 and there isn't http example for it. I'm in wrong?

Alessandro,
Try to change size directly on pubsubclient configuration file . Btw also look at mqtt error codes , you can find a good tables of it on lwmqtt arduino repository

Alessandro,
Try to change size directly on pubsubclient configuration file .

WOW! Now with 1024 (with 2048 esp32 go in bootloop) on file pubsubclient.h I have this:
Error, client state: 5
A big step forward! :P

@alexodus the esp32-ssd1306 example is HTTP, probably could use a rename ^_^.

SOLVED! I had only set MQTT_MAX_PACKET_SIZE more than 324 into pubsubclient.h (I don't like it but I think this is a platformio problem). MQTT_CONNECT_UNAUTHORIZED happen because I modified the private key for test :P

Best solution for platformio: add this line on platformio.ini file
build_flags = -DMQTT_MAX_PACKET_SIZE=384
see this issue on pubsubclient library

Ahh, thank you @alexodus may be worth adding to the readme, if you want me to do it, happy to accept an issue, if you want to make a PR, happy to review.

Added note to the readme, thanks for the errata!

authors: Gior80 alexodus gguuss


text:
Pubsub: Invalid resource name given

When I try to follow the example code as given here:
(NOTE: I altered the hardcoded strings here to match the ones I need)
ProjectTopicName topicName = ProjectTopicName.of("my-project-id", "my-topic-id");
Publisher publisher = null;
List<ApiFuture<String>> messageIdFutures = new ArrayList<>();

try {
  // Create a publisher instance with default settings bound to the topic
  publisher = Publisher.newBuilder(topicName).build();

  List<String> messages = Arrays.asList("first message", "second message");

  // schedule publishing one message at a time : messages get automatically batched
  for (String message : messages) {
    ByteString data = ByteString.copyFromUtf8(message);
    PubsubMessage pubsubMessage = PubsubMessage.newBuilder().setData(data).build();

    // Once published, returns a server-assigned message id (unique within the topic)
    ApiFuture<String> messageIdFuture = publisher.publish(pubsubMessage);
    messageIdFutures.add(messageIdFuture);
  }
} finally {
  // wait on any pending publish requests.
  List<String> messageIds = ApiFutures.allAsList(messageIdFutures).get();

  for (String messageId : messageIds) {
    System.out.println("published with message ID: " + messageId);
  }

  if (publisher != null) {
    // When finished with the publisher, shutdown to free up resources.
    publisher.shutdown();
  }
}

It raises an exception on:
ApiFutures.allAsList(messageIdFutures).get();
Exception:
java.util.concurrent.ExecutionException: com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid resource name given (name=testResource) Refer to https://cloud.google.com/pubsub/overview#names for more information.

When I publish a message through the web console, it works as intended and my code (based on the example subscriber code) can read it out correctly.
My resource name matches with what I see in the console though:
projects/testProject-2013370/topics/testResource

Project info:
Java version: 1.8.0_131
Pubsub dependency:   'com.google.cloud:google-cloud-pubsub:1.31.0'

Hi @DylanMeeus Sorry I wasn't able to repro this issue.
Can you make sure in this case, your topicName is created by:
ProjectTopicName topicName = ProjectTopicName.of("testProject-2013370", "testResource");

If that still does not help, can you try
publisher = Publisher.newBuilder("projects/testProject-2013370/topics/testResource").build();

and see if that works?

I am getting the same issue. I think it's when I end up pulling.
Jun 05, 2018 3:13:25 PM com.google.cloud.pubsub.v1.StreamingSubscriberConnection$1 onFailure
SEVERE: terminated streaming with exception
com.google.api.gax.rpc.InvalidArgumentException: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid resource name given (name=random). Refer to https://cloud.google.com/pubsub/overview#names for more information.
	at com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:49)
	at com.google.cloud.pubsub.v1.StreamingSubscriberConnection$1.onFailure(StreamingSubscriberConnection.java:241)
	at com.google.common.util.concurrent.Futures$4.run(Futures.java:1123)
	at com.google.common.util.concurrent.MoreExecutors$DirectExecutor.execute(MoreExecutors.java:435)
	at com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:900)
	at com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:811)
	at com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:675)
	at com.google.common.util.concurrent.SettableFuture.setException(SettableFuture.java:53)
	at com.google.cloud.pubsub.v1.StreamingSubscriberConnection$StreamingPullResponseObserver.onError(StreamingSubscriberConnection.java:178)
	at io.grpc.stub.ClientCalls$StreamObserverToCallListenerAdapter.onClose(ClientCalls.java:419)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
	at io.grpc.internal.CensusStatsModule$StatsClientInterceptor$1$1.onClose(CensusStatsModule.java:684)
	at io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:41)
	at io.grpc.internal.CensusTracingModule$TracingClientInterceptor$1$1.onClose(CensusTracingModule.java:391)
	at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:475)
	at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:63)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.close(ClientCallImpl.java:557)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl.access$600(ClientCallImpl.java:478)
	at io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:590)
	at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
	at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:123)
	at java.util.concurrent.Executors$RunnableAdapter.call$$$capture(Executors.java:511)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
	at java.lang.Thread.run(Thread.java:748)
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid resource name given (name=random). Refer to https://cloud.google.com/pubsub/overview#names for more information.
	at io.grpc.Status.asRuntimeException(Status.java:526)
	... 21 more

TopicAdminClient.create().createTopic(topicName) topicName is Test
ProjectTopicName.of(ServiceOptions.getDefaultProjectId(), topic) again, topic is Test
SubscriptionAdminClient.create().createSubscription(name, topicName, PushConfig.getDefaultInstance(), 0) name is apparently random
It kinda sorta seems to imply that random is just an invalid format for a subscriber name, however, if names have to be in a particular format it would be nice if the documentation specified this... The link provided in the exception mentions NOTHING about subscription name formatting.

@hzyi-google I will test this asap.
@MrThreepwood Names do have to adhere to a certain format. As you have no doubt experienced as well, the URL the exception refers you to is useless. (for which I have made #3333 , which was swiftly resolved so props for that!)
The URL you should be looking at, as it turns out, is this one: https://cloud.google.com/pubsub/docs/admin#resource_names
Neither "random" nor "testResource" violate these restrictions.

I figured out what my actual issue was. I was accidentally passing the string "random" to Subscriber.newBuilder() rather than he ProjectSubscriptionName.
I now have it up and working. So obviously the full name should end up being something along the lines of projectId/name. If using the builder with the string name parameter it's not obvious what the actual issue is by the error provided.
Thanks for the quick reply.

@hzyi-google
I have tried it with the second builder you suggested - which worked.
After that, I have also found the problem with my original code.. rather than using the "topic name" I had used the "topic id" in this line: publisher = Publisher.newBuilder(topicName).build();.
:man_facepalming:
Thanks for the alternative method, that eventually led me to the solution!

Glad it helped.

authors: DylanMeeus MrThreepwood hzyi-google

repo: GoogleCloudPlatform/google-cloud-java


text:
Remove search API

Looks like we're changing things around and search won't be it's own API surface.
Can we chop from gcloud-node? (It was a private alpha, so the impact to gcloud-node users should be relatively minimal). If there are concerns about timing, let's discuss here.

Where does/will the search API live now? I can't find any documentation about it for Node in the flexible environment.

Does anyone have any word on the Search API? I really need it for my current project that I'm migrating over from the Java app engine side of things. It's the one thing holding me back from going full nodejs.

The Search API was in an alpha state when we supported it. I can't be certain if it will resurface in our API someday, but for now, you should be able to roll back to gcloud version 0.31.0 to get through to it.

Curious why it was removed if there's no plan to provide an alternative? If the code from version 0.31.0 still works, why not leave it in the current version (with warnings?) until the new API comes along?
I suppose there's nothing wrong with pulling the code out into a companion library that depends on this one.

authors: jgeewax lostpebble modeswitch stephenplusplus

repo: GoogleCloudPlatform/google-cloud-node