text:
🛑 Ader is down

In 08a015b, Ader (https://www.ader-paris.fr/en/home) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Ader is back up in 6038322 after 13 minutes.

source: gharchive/issue

repo: divtiply/artupptime

license: MIT

license_type: permissive

license_source: github-api


text:
🛑 Darz is down

In 6af79c4, Darz (https://darz.art/en) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Darz is back up in 4f294a5 after 4 minutes.


text:
opentherm-thermostat/boiler-target-temperature/get gets status "nan" after a Home Assistant Restart

I Don't know if it is regarding home assistant, your ino, or the boiler.
But after a home assistant restart (not the mqtt broker) the payload of opentherm-thermostat/boiler-target-temperature/get gets the value "nan". The boyler doesn't responds to commands any longer. The only fix is to restart the esp8266.
I tried to get it to work by sending idle command followed by a heat command and some other things, but nothing helps except an esp reboot.
Maybe a reboot command through mqtt could be a solution?

Was an issue with the MQTT Server that would interrupt the messages, send garbage to the esp8266


text:
一日のプレイ曲数をcsvでグーグルドライブに保存

ボルテの公式サイトで確認できるプレー回数のように自分が一日にプレイした曲数が保存できる
機能があればうれしいです
A列　　　　　　　B列
2024年1月8日　　20
2024年1月9日　　50
2024年1月10日　 60
こんな感じで保存されてると嬉しいです

v.1.0.20で対応しました。
playcount.csvというファイル名で、以下のように自動保存されます。

対応ありがとうございます！


text:
Feature Need: Support backgroup async repeat task. just like fastapi_utils.tasks.repeat_every

Feature Need: Async django should make repeat task much more easy.
I like to use fastapi_utils.tasks.repeat_every, so easy and doc is here:
https://fastapi-utils.davidmontague.xyz/user-guide/repeated-tasks/
from fastapi import FastAPI
from fastapi_utils.tasks import repeat_every

app = FastAPI()

@app.on_event("startup")
@repeat_every(seconds=60 * 60)  # 1 hour
def remove_expired_tokens_task() -> None:
      pass

For Django, I found django-celery-beat and django-q and so on, but they are too heavy for depending a sub-system which name broker.
May be django-extensions can support this feature when running in async mode. (ASGI Django)
ref:  django # 34447

async task is support in fastapi with starlette:
https://www.starlette.io/background/
from starlette.applications import Starlette
from starlette.responses import JSONResponse
from starlette.background import BackgroundTasks

async def signup(request):
    data = await request.json()
    username = data['username']
    email = data['email']
    tasks = BackgroundTasks()
    tasks.add_task(send_welcome_email, to_address=email)
    tasks.add_task(send_admin_notification, username=username)
    message = {'status': 'Signup successful'}
    return JSONResponse(message, background=tasks)

routes = [
    Route('/user/signup', endpoint=signup, methods=['POST'])
]

app = Starlette(routes=routes)

repo: django-extensions/django-extensions

license: mit

license_source: bigquery


text:
runserver_plus command, pm (postmortem) mode does not work with python 3

Issue is on Line 175 in management/commands/runserver_plus.py:
print >>> sys.stderr, "Exception occured: %s, %s" % (exc_type, exc_value)
This syntax does not work with Python 3. To make it work, we need to do:
from __future__ import print_function
....
.....
print("Exception occured: %s, %s" % (exc_type, exc_value), file=sys.stderr)

Should I send a PR?

Sorry for the extremely late reply.
Yes please send a PR to fix this if it's still necessary.

authors: eskhool trbs


text:
Do not access the session unless needed

If the session wasn't accessed we do not need to modify the response.
Session get accessed lazily if someone accesses request.user or
request.session. In those cases Django will add cookie to the
Vary-On HTTP header. This will prevent browsers from caching the
results.
Furthermore, accessing the user when not needed will cause unwanted
IO on request, that don't need authentication or vary content based
on users.
We use the same behavior Django implements in the authentication
middlware, where a user is a lazy object and will only be fetched
if needed. We simply overwrite the user and add another setup
hook.

Codecov Report

Merging #509 (e697759) into master (bc8be46) will increase coverage by 0.02%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##           master     #509      +/-   ##
==========================================
+ Coverage   99.24%   99.27%   +0.02%
==========================================
  Files           7        7
  Lines         133      138       +5
==========================================
+ Hits          132      137       +5
  Misses          1        1

Impacted Files
Coverage Δ

hijack/middleware.py
100.00% <100.00%> (ø)

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

source: gharchive/pull-request

authors: codecov-commenter codingjoe

repo: django-hijack/django-hijack


text:
Escape HTML output

As per Django's documentation

Clearly, user-submitted data shouldn’t be trusted blindly and inserted
directly into your web pages, because a malicious user could use this kind of
hole to do potentially bad things. This type of security exploit is called a
Cross Site Scripting (XSS) attack.

To avoid this problem, you have two options:

One, you can make sure to run each untrusted variable through the escape
filter (documented below), which converts potentially harmful HTML characters
to unharmful ones.

Hi Klaas - thanks so much for raising the issues and suggesting PRs, we appreciate it.  I would recommend you test against v3 which is in beta (branch is release-3-x).
There are a lot of changes coming in v3.  You'd be very welcome to submit PRs in either v2 or v3, but please check out the contribution guide first.  v3 is going to soon become the mainline version, with v2 supported for some time for bug fixes.

This particular PR does not apply on release-3-x; given its nature, might I suggest merging it into master anyway?

@matthewhegarty anything I can do to move this PR and the others forward?

I'll have to take a look when I get some time - maybe at the weekend.  Thanks for your contributions.

Thanks again for submitting this.  I fixed the isort error and updated changelog.  If you are ok with that I will merge.

Coverage increased (+0.001%) to 98.395% when pulling 8285b44bdd25d92cf8b455a67d2781434d9338c5 on vanschelven:escape-html-output into eb27821a899a651fd11fe92a5b88955eed35ae0c on django-import-export:main.

LGTM, feel free to merge

@matthewhegarty any chance you could release 2.8.1 to include this PR (and a few others) in a proper release?

2.9.0 released

authors: coveralls matthewhegarty vanschelven


text:
Wrong parameters passed to Applicator with Oscar 1.1

In the recent Oscar version 1.1, Applicator expects next parameters - basket, user, request:
class Applicator(object):

    def apply(self, basket, user=None, request=None):

Per https://github.com/django-oscar/django-oscar/blob/master/src/oscar/apps/offer/applicator.py#L19.
However, Applicator in django-oscar-paypal sends:
# Re-apply any offers
        Applicator().apply(basket, self.request.user, self.request)

Per https://github.com/sasha0/django-oscar-paypal/blob/master/paypal/express/views.py#L228

bump!

thanks! it works!

Merging #101 should've fixed this. Thanks for the good issue report.

authors: feado fjsj maikhoepfel sasha0

repo: django-oscar/django-oscar-paypal


text:
Add a multi_option value type. This will create a new table for the m…

Fixes https://github.com/django-oscar/django-oscar/issues/1739
I took inspiration from https://github.com/django-oscar/django-oscar/pull/879 but tried to implement it in such a way that it would be backwards compatible.
It's not a particularly elegant solution. There's now both a ForeignKey and ManyToManyField to catalogue.AttributeOption, but ProductAttributeValue isn't a particularly elegant model to begin with.
I initially thought I would add a ManyToManyField and then make the option field actually use it, but doing so would make backwards compatibility really difficult.
To do
I am willing to do these, but I would appreciate some input before proceeding.

Update the initial migration.
Documentation.
Tests

Thanks,
John

As per discussion https://github.com/django-oscar/django-oscar/pull/1745#issuecomment-106025693, it might be better to make value return a QuerySet for instead of a Manager for multi-valued fields. You could still get the manager if you really need it by accessing ProductAttributeValue.value_multi_option.

One thing you can do with an option is attribute.value = 'Option' and it will automatically look up the option and make sure it's valid: will raise a DoesNotExist exception if you try to set it to an invalid option.
With this, you can't do something like attribute.value = ['Option 1', 'Option 2', 'Option 3'].
If this functionality is necessary, I could work it in.

After looking at both, I prefer this to #879
and I do think it's slightly better returning a QuerySet :)
re setting the value... I was surprised that I didn't need to override anything in _set_value for eg the Product update form to work, but I guess to be able to do attribute.value = ['Option 1', 'Option 2', 'Option 3'] programmatically then we'd need handling of the m2m field
You haven't removed the unique_together from AbstractProductAttributeValue in this PR, can it work without doing that?
I assumed we need to, in which case we should ad back in a custom validate_unique conditionally for single-value options

Yes, unique_together should remain. The m2m relationship is on an entirely different table and subject to its own constraints.

Looks like this adds some lines that are too long (according to flake8). I'll see if I can wrap them.

Is there anything else I need to do with this?
@maikhoepfel Any thoughts?

Finished this off and merged in #2309.

authors: anentropic john-parton solarissmoke

repo: django-oscar/django-oscar


text:
Can you provide examples

This is very close to a project we've been building.
Could someone post screenshots of what this implementation looks like?
It seems to be a django cms implementation of the salesforce api but I can't see what it creates and rather than going through the whole installation and configuration to find out it doesn't do it.
Please post photos of this installed.
I'm curious to help out and debug this as it seems very close to what I was planning.

This is a database backend and text examples of queries and responses are more typical than screenshots in a documentation for similar products, but I understand you. The documentation for Python, Django or SQL databases looks like a text with many screenshots, but it is internally a formatted text. Probably both simple usual examples and also short examples of the most complex supported queries should be added. If you agree I rename "screenshots" to "Can you provide examples".

Some advanced examples of more complicated supported Django querysets are here. I'm mot sure if you are interested in simple or advanced examples.

This is not a CMS in any sense of the word. I think we're pretty clear that this is a database backend for Django.

authors: datatalking hynekcer philchristensen

repo: django-salesforce/django-salesforce


text:
Fixed #30599 -- Prevented ManifestFilesMixin.read_manifest() from silencing errors other than FileNotFoundError.

ticket 30599

Thanks @zeyneloz!
It'd be great if you could add regressions tests to make sure PermissionError are surfaced and FileNotFoundError are silenced.

@zeyneloz Thanks for taking this on! You rock!

Thanks for the comments. I added regression tests.

@zeyneloz Thanks for this patch :+1: I pushed minor edits and moved test that works without this patch to a separate commit.

authors: charettes felixxm mblayman zeyneloz

repo: django/django


text:
Fixed #31804 - added support for parallel database setups

Trac ticket number
ticket-31804
Branch description
Adds support for parallel test setup
Performance
Speed is nearly the same on my machine (M1 Pro 16GB, 512GB), however, the ticket mainly mentioned slowness issues on Windows, and I will appreciate someone testing on Windows.
Checklist

[x] This PR targets the main branch.
[x] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[x] I have checked the "Has patch" ticket flag in the Trac system.
[ ] I have added or updated relevant tests.
[ ] I have added or updated relevant docs, including release notes if applicable.
[ ] I have attached screenshots in both light and dark modes for any UI changes.

@sarahboyce I will appreciate some support about testing this PR

I'm not sure if cloning is IO-bound or CPU-bound

I think cloning is more IO-bound, since really it's just copying data between DBs (rather than doing much transformation in the middle). Either way, the IO / CPU bound-ness is more for the DB engine itself, rather than Django. Most of the time, Django is sitting waiting on messages from a socket, where threads is definitely the right way to handle it.

I'm not sure if cloning is IO-bound or CPU-bound

I think cloning is more IO-bound, since really it's just copying data between DBs (rather than doing much transformation in the middle). Either way, the IO / CPU bound-ness is more for the DB engine itself, rather than Django. Most of the time, Django is sitting waiting on messages from a socket, where threads is definitely the right way to handle it.

Glad to hear, I missed removing the event from a previous implementation that had the event and

Hello Ahmed. Appreciate you picking this up. It's something that I've always wanted to revisit
A couple of questions from my side:

Could we specify parallelization unique to each db? The original idea was to utilize the generated dumps for cloning db's but doing so in parallel instead of serially. (i.e using pg_dump, mysqldump, oracle's import/export API)
Are threads better here than processes? I'm not sure if cloning is IO-bound or CPU-bound. Again this is related to the first question more. If indeed we're IO-bound, your threading approach looks good.

Thanks for the kind words, regarding your questions, Can you elaborate on how the dump approach would be better in our case?
The database cloning stuff is mostly requests going to the databases, nothing is actually happening on our end, but on theirs, I'm not sure about sqlite3, but I guess since sqlite3 doesn't have a server, we may end up not improving tests performance on sqlite3, but I think this is ok

@RealOrangeOne thanks for the comment, you're correct, I pushed

authors: LeOndaz RealOrangeOne


text:
Fixed #28863 -- Fixed filter on annotation that contains Q.

https://code.djangoproject.com/ticket/28863

LGTM apart from SQL changes. But I can see why the parens might always be needed in order to ensure the SQL always compiles.

buildbot, test on oracle.

The error for the new test on Oracle django.db.utils.DatabaseError: ORA-00923: FROM keyword not found where expected.

Using boolean type in SELECT is not supported on Oracle. The only way to emulate this is to use CASE ... WHEN .. syntax:
SELECT COUNT(*)
FROM (
    SELECT
        "EXPRESSIONS_COMPANY"."ID" AS Col1,
        (CASE WHEN "EXPRESSIONS_COMPANY"."NUM_EMPLOYEES" > :arg1 THEN 1 ELSE 0 END) AS "NUM_EMPLOYEES_CHECK"
    FROM "EXPRESSIONS_COMPANY"
    WHERE
        (CASE WHEN "EXPRESSIONS_COMPANY"."NUM_EMPLOYEES" > :arg1 THEN 1 ELSE 0 END) > :arg1) = :arg0
    GROUP BY
        "EXPRESSIONS_COMPANY"."ID",
        (CASE WHEN "EXPRESSIONS_COMPANY"."NUM_EMPLOYEES" > :arg1 THEN 1 ELSE 0 END)
    ) subquery

If besides this issue, the 2.0 release is ready later today, I don't think I'll block the release on it. It can always be fixed in a later bug fix release.

We can skip this test on Oracle, I'm fine with that.

Thanks @sir-sigurd @tim-schilling @felixxm :)

authors: adamchainz felixxm sir-sigurd timgraham


text:
Add an automatic gain reduction mechanism when overload is detected

While working through DAB, I found that signal overload is a significant concern. This probably applies to HD Radio as well but I hadn't noticed to date.  What is happening on DAB is that overload causes the I/Q samples to max out at -1/1 (0/255), and the welle.io backend requires a certain amount of deviation in the signal to determine it's "sync" state and actually start processing the data.
For welle.io, at least, an overloaded signal may cause it to effectively lock up in an infinite loop waiting for a "sync" that may never occur. For now, based on the sample input I have (which is overloaded), I changed welle.io to not be so picky and it's moving along, but I think that tweak needs to be undone moving forward.
Implementation should be pretty simple; when converting the pair of 8-bit samples into Complex I/Q samples, if either input byte is 0 or 255 the signal can be assumed to be overloaded. Add a simple decrease_gain() function to the device implementation that "ticks" it down one notch from whatever it is currently set to.  I feel there is no need or valid use case for the opposite operation -- detecting headroom in the signal and automatically increasing gain. That operation would make this more complex than it needs to be, this is not trying to be software-based "automatic gain control", it's trying to be something to ensure that noise spikes on very wide channels like DAB and/or HD Radio don't prevent those demodulators from doing their thing.
I feel like this should be something that the user should opt-in on as opposed to being the default and forcing them to opt-out. Option might be attached to the device settings if it appears to have any value for FM/WX, but I think it might be better served attached to HD Radio / DAB settings.

This project will be archived, closing Issue as unresolved.


text:
🛑 Writeguard is down

In d461ef0, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in 265baac.


text:
🛑 Writeguard is down

In ab66d79, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in ae6df14.


text:
🛑 Writeguard is down

In 16bb26e, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in 4bdf0c1.


text:
🛑 Writeguard is down

In 992e657, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in 6ec7efb.


text:
🛑 Rampa_Blato is down

In 256bc6b, Rampa_Blato (http://rampa.ddns.net:8001/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Rampa_Blato is back up in 4cf20cf.


text:
[WIP] Git support: first steps

This is a WIP approach to support to new features.
Based on a label (or maybe comment, but label is easier for now as authorization is implicitly given), support the following "actions":

"bot-rebase" (label) -> "git clone" -> rebase to latest master -> "git push" (if no failure happened)
"bot-squash" (label) -> "git clone" -> squash all commits -> "git push"

Imho both are very useful features

rebase can be used to re-trigger Jenkins
squash can be used by reviewers to automatically squash everything before applying "auto-merge"

This PR implements only "rebase", but adding "squash" later won't be difficult
What's there?

Implemented SmartGit server protocol (only the needed parts, of course)
Added unittest abstraction over SmartGit server

This means that we can fully handle git push commands as "faked" server, inject the requests and control the responses like we already do for other mocked APIs.
> git push -v --force http://0.0.0.0:9006/aa/bar HEAD:Issue_8573

The command will be handled by the mocked backend :)
// first request by the git client
// server returns a list of all refs it knows
gitInfoRefs = (string user, string repo) {
    "requesting refs".writeln;
    return [
        GitInfoRef("6ed2e15cf4a35e274adb8385b3ee8125326509f9", "refs/heads/foo"),
        GitInfoRef("6e79d22fdfda446601d969ce77e406b9a5506de9", "refs/heads/Issue_8573"),
        GitInfoRef("6e79d22fdfda446601d969ce77e406b9a5506de8", "refs/heads/master"),
    ];
};

// git client "reports" new refs to the server
gitReportRefs = (ClientReq clientReq) {
    clientReq.writeln;
    // GitClientRequest([], [Command(update, "6e79d22fdfda446601d969ce77e406b9a5506de9", "e4afacadb8a291f51444e948dd7b53592e799dbe", "refs/heads/Issue_8573")])
    return [
        GitReportRef(GitReportRef.status.ok, "refs/heads/Issue_8573")
    ];
};

What's missing?

[ ] fix reason for Vibe.d blocking (not sure about this, but std.process.wait seems to block the entire Vibe.d thread  even if we are in a new fiber (e.g. runTask), on a worker thread (e.g. runWorkerTask) and when spawned with std.concurrency`)
[ ] add simple, "immutable" test repository with a couple of small PRs
[ ] extensive testing (on GH with dummy repo)
[ ] remove hard-coded gitURL
[ ] add support for authentication (this requires saving the bot's password or private key on heroku)
[ ] final cleanup

Implemented SmartGit server protocol (only the needed parts, of course)

That seems like overkill. Why not just use a bare repo?

That seems like overkill. Why not just use a bare repo?

Why? I explicitly want to test what the bot will send to GitHub, so mocking the API seemed very reasonable.
Btw this is the way we test the other bot actions as well and force rebasing is a potentially dangerous action, hence I want to have an extensive test coverage before testing this in the wild.

Why?

Well, it tests implementation detail of third-party software, relies on the git implementation used to behave as it does now, and it would make it overly onerous to write tests for more complicated functionality in the same vein.
It would make sense if dlang-bot had its own git client and SmartGit protocol implementation, but it doesn't - it uses the git shell commands, so I think it would be more logical to test their effects, not implementation details.
IMHO. shrug

It would make sense if dlang-bot had its own git client and SmartGit protocol implementation, but it doesn't - it uses the git shell commands, so I think it would be more logical to test their effects, not implementation details.

How would you test the effect of git push?

You can push to a bare repo on the filesystem.

What do you guys think about using comments, instead of labels for this? Specifically about squashing, it would be much better if a reviewer could manually specify a commit message.

authors: CyberShadow ZombineDev wilzbach

repo: dlang-bots/dlang-bot


text:
std.variant.Algebraic test use case

bearophile_hugs reported this on 2010-10-10T14:42:54Z
Transfered from https://issues.dlang.org/show_bug.cgi?id=5037
CC List

andrei (@andralex)

Description
Created attachment 781
One memory/runtiume performance test for the compiler/binary for the flatten

This isn't a bug report.

In dmd 2.049 the docs for std.variant.Algebraic state:

> BUGS: Currently, Algebraic does not allow recursive data types.
> They will be allowed in a future iteration of the implementation.

Once that limit is lifted, the following problem may be used to see if the API
of Algebraic is well designed. The problem is, given an arbitrary nested list
of values and sublists:

[[1], 2, [[3,4], 5], [[[]]], [[[6]]], 7, 8, []]

To flatten it fully into this result:
[1, 2, 3, 4, 5, 6, 7, 8]

This is a solution that minimizes heap activity, it doesn't use Algebraic but
it uses a tagged union:

import std.stdio: writeln;
import std.conv: to;

struct TreeList(T) {
    union {
        TreeList!T[] arr;
        T data;
    }
    bool isArray = true;

    static TreeList!T opCall(Types...)(Types items) {
        TreeList!T result;

        foreach (i, el; items)
            static if (is(Types[i] == T)) {
                TreeList!T item;
                item.isArray = false;
                item.data = el;
                result.arr ~= item;
            } else
                result.arr ~= el;

        return result;
    }

    TreeList!T flatten() {
        TreeList!T result;

        if (this.isArray)
            foreach (el; this.arr)
                result.arr ~= el.flatten().arr;
        else
            result.arr ~= this;

        return result;
    }

    string toString() {
        if (isArray)
            return to!string(arr, "[", ", ", "]");
        else
            return to!string(data);
    }
}

void main() {
    alias TreeList!(int) L; // 12 bytes if T is int
    auto l = L(L(1), 2, L(L(3,4), 5), L(L(L())), L(L(L(6))), 7, 8, L());
    writeln(l);
    writeln(l.flatten());
}

Well implemented Algebraic types are supposed to allow a shorter, simpler and
equally runtime/memory-efficient solution to this problem (the efficiency may
be tested with the large list literal in the attach file).

!!!There are attachements in the bugzilla issue that have not been copied over!!!

andrei (@andralex) commented on 2016-10-14T13:14:16Z
We have limited support for self-referential structures now, so we should be able to work on this.

repo: dlang/phobos


text:
Posix.mak: cleanup EXTRA_MODULES_INTERNAL + add stdiobase to internal

This list was pretty hard to read. Hence:
First commit:

only use add_prefix if actually saves space (i.e. don't use add_prefix for single words)
sort alphabetically
indent properly
add add_prefix where it improves readability (e.g. unicode

Second commit (after cleanup): add stdiobase to the list internal modules
A PR to dlang.org to exclude it from the menu will follow soon.

A PR to dlang.org to exclude std.stdiobase from the menu will follow soon.

-> https://github.com/dlang/dlang.org/pull/1579

@CyberShadow as you have just removed std.stdiobase from DDox, here's the respective PR for DDoc. It seems like there's no (free) Makefile Guru expect for you around ;-)


text:
Consider using colors.json instead

I notice that this imports ~/.cache/wal/colors
However this file does not include the "special" section found in colors.json which includes background and foreground.
It would be nice if this theme would import the json and used those properties if they exist to better match the actual background and foreground of the system when applied with wal.
Thank you!

I see your point, and I might change this one day, but in the near future probably I won't. If you want, you can send me a PR.
That said, I don't understand what do you mean with 'match the actual background and foreground', could you explain it to me?

If you want, you can send me a PR.

It seems like it might be a straightforward change, I'll look into it

That said, I don't understand what do you mean with 'match the actual background and foreground', could you explain it to me?

Sure!
As mentioned, the extension looks for ~/.cache/wal/colors. By default these only include colors 1-16
https://github.com/dlasagno/vscode-wal-theme/blob/ccc84e4944aa7b7512bc7f3c8ffb7cb98c906e78/src/extension.ts#L8
Here is an example of my wal colors file
› cat .cache/wal/colors
#1B2B34
#E15A60
#99C794
#FAC863
#6699CC
#C594C5
#5FB3B3
#C0C5CE
#65737E
#E15A60
#99C794
#FAC863
#6699CC
#C594C5
#5FB3B3
#D8DEE9

However, pywal supports a theme to have base-16 colors defined, like background, foreground and cursor. These values are typically used for the background and foreground of the terminal, and other tools like polybar which can use pywal themes. Here is a predefined pywal theme that has these properties
Below is an example of my colors.json which references these properties
› cat .cache/wal/colors.json
{
    "wallpaper": "None",
    "alpha": "80",

    "special": {
        "background": "#242B2E",
        "foreground": "#D4D4D4",
        "cursor": "#C0C5CE"
    },
    "colors": {
        "color0": "#1B2B34",
        "color1": "#E15A60",
        "color2": "#99C794",
        "color3": "#FAC863",
        "color4": "#6699CC",
        "color5": "#C594C5",
        "color6": "#5FB3B3",
        "color7": "#C0C5CE",
        "color8": "#65737E",
        "color9": "#E15A60",
        "color10": "#99C794",
        "color11": "#FAC863",
        "color12": "#6699CC",
        "color13": "#C594C5",
        "color14": "#5FB3B3",
        "color15": "#D8DEE9"
    }
}

Note that #242B2E (along with the other "special" colors) are not present in ~/.cache/wal/colors
Now in template.ts it uses color0 and color1 for the background/foreground.
https://github.com/dlasagno/vscode-wal-theme/blob/ccc84e4944aa7b7512bc7f3c8ffb7cb98c906e78/src/template.ts#L104-L105
To be more true to a pywal theme one might expect the VSCode theme to use the defined backround and foreground colors
To summarize, the background for VSCode with this theme would be based on #1B2B34 and not #242B2E making VSCode not match the rest of the system that uses #242B2E as a background color.
Hopefully I provided enough context! Please let me know if there are further clarifications I can make :slightly_smiling_face:

I know that 6 months passed since the issue was opened, but I'm finally looking back into this since I'm planning an update for the extension anyway.
Before implementing this I was wondering what colors 0 and 7 would be used for after being replaced by background and foreground. @chrishoage, could you explain to me how they're used in a setup like yours?

They're generally not in my setup. I would imagine colors.json would be used to see if background and foreground are set, and otherwise fall back to the colors pulled from the colors object in colors.json
Looking at some image-generated themes from pywal background and foreground are set to color0 and color7 respectively.
However, pywal still allows these to be set independently from the normal base16 colors, where "black" (color0) can be a different color in the theme for background.
Please let me know if there are further clarifications I can help with!

At the moment to generate the theme I pass an array of 16 colors to a template. Do you think that setting colors 0 and 7 to the ones specified for background and foreground(if present) would achieve the desired outcome?

That's a good question, I'm uncertain with out seeing it. I think the desired outcome would be things that are "background" colors (e.g. the main editor background, perhaps the sidebar background, statusbar background) would all get the background color. Foreground things (like text, selection, etc) would get the foreground color.
Another thing that might be worth considering is to pull the alpha value from the colors.json and use it for the alpha value for the background colors.

Version 1.1.0 is now available and uses the custom background and foreground entries of colors.json.
In the end, I simply changed colors 0 and 7 with background and foreground respectively.

authors: chrishoage dlasagno

repo: dlasagno/vscode-wal-theme


text:
Error reporting broken in v1.3.2

Eg. remove the ; from the end of a variable and get this error using 1.2.4:
/app/utility/variables:47: top-level variable binding must be terminated by ';'

but with 1.3.2 I get this:
 [gulp-sass] [object Object]

Yes, that is what you get when you or some other code "toStrings" the object. I think the issue may be that the error object doesn't have Error in its prototype chain, which I think would be a node-sass issue.
I hacked this into my error handler in gulp very quickly. There was no good reason to use a try/catch block, but I was being lazy:
    try {
      errString = JSON.stringify(err, false, ' ');
    }
    catch (myErr) {
      errString = err.toString().trim();
    }

I don't know what the "right" solution is, but I suspect it should be raised on node-sass, hopefully before 2.0 is frozen, if indeed the error objects coming out don't have a useful toString implementation in their chain.

This has been resolved in 2.x. Please try it and report back. If we don't hear back, we'll assume it's been resolved and close this issue.
npm install dlmanning/gulp-sass#2.x

Just tried it. Worked great--Thank you!

Can't figure out how to fix it, any idea?

I'm… gonna close now. As you seem to confirm that we've indeed resolved this issue

authors: Ariix Snugug laurelnaiad scerelli valdelama

repo: dlmanning/gulp-sass


text:
Integrate nearest neighbour search for translating input emission inventory onto contrail grid

In OAC v2.9.0, the same contrail grid interpolation will be used as in AirClim 2.1.

Added in bde7648

repo: dlr-pa/oac


text:
Fix printf format string and arguments

Fixes compile issues as reported in #559.

thx @joakimfors  for your pr. i will review it soonish

thx @joakimfors

authors: MalloZup joakimfors

repo: dmacvicar/terraform-provider-libvirt


text:
Standardized coefs

This is a follow up for #135 about what I would require the dataframe used for fitting.
Mixed models are (although too slowly) becoming the default and one of the most widely used methods in psychology. However, there are certain limitations for users and reviewers that in my opinion slow the transition. In particular, users often are not sure about how and what to report in their manuscript (based on the output provided for instance by R), leading to a high variability in the reported information (with often missing or unexact stuff). Moreover, we are often required (by reviewers/supervisors) to provide additional metrics, which can be statistically not relevant or debatable or useless or wrong for mixed models (cf the long debate around p values, pseudo-R² and so on).
In this context, I am trying to create small convenience functions to ease data analysis in the psychology field, that takes a result (for example a mixed model) and returns a formatted textual output, with the goal of standardizing the reports (with, by default, the inclusion of "best practices").
I started implementing this for GLMs (see here, it's in very WIP state) and would like to do it for GLMMs. One of the ingredients I'd like to include is the possibility of returning standardized coefficients, to allow their categorization as index of "effect size" with a set of rules of thumb (such as Cohen's 1988 or their own).
Long story short, I've seen that one method of computing such std coefs is basically to directly transform the coefficient (and divide it by the SD of the outcome variable). However, I believe that this method provides misleading values in some cases (such as interaction effects). The other method is to standardize the data beforehand, refit the model and extract the coefs of this model. The main limitation is that it is computationally costly (as one have to fit the model two times), but in most use-cases in psychology, frequentist models are quite fast to compute, so this additional time is acceptable.
This is why I would need the original dataframe, to be able to standardize it and update (re-fit) the model to get "standardized coefs". Is there any other way of computing them using model matrices? Thank you

@nalimilan How would it work?

I was going to mention that if you just need the model matrix for the fixed-effects terms that is stored, as a plain matrix in transposed form, as part of the second last component of the trms field.  I could quite easily add an extractor for that.

@nalimilan How would it work?

I don't know precisely, but that's clearly an issue which is going to be raised with lots of other model families, so we need a general solution. Better discuss that in a StatsModels issue.

If it will help I have just committed changes to the "getproperty" branch so that you can retrieve the fixed-effects model matrix as
julia> m.X
30×1 Array{Float64,2}:
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0
 1.0

and the response vector as
julia> show(m.y)
[1545.0, 1440.0, 1440.0, 1520.0, 1580.0, 1540.0, 1555.0, 1490.0, 1560.0, 1495.0, 1595.0, 1550.0, 1605.0, 1510.0, 1560.0, 1445.0, 1440.0, 1595.0, 1465.0, 1545.0, 1595.0, 1630.0, 1515.0, 1635.0, 1625.0, 1520.0, 1455.0, 1450.0, 1480.0, 1445.0]

I'll close this issue for now in the hope that a more general solution will be found (see here). Thanks a lot for your help!

authors: DominiqueMakowski dmbates nalimilan


text:
Update metadata

License: Update year
README:

drop PkgEval since it has been deprecated for 0.7/1.0
Drop reference for lmm and glmm as documentation favors using the fit with the type
Drop the installation quickstart since package is registered

Coverage remained the same at 89.559% when pulling a6a763c950d56a957bfdb9cbea0c9571f19abd91 on Nosferican:Update-Metadata-to-Julia-1 into 380d3d1428df3356f15cb1093b164eecbfc0ed66 on dmbates:master.

Coverage remained the same at 89.559% when pulling a6a763c950d56a957bfdb9cbea0c9571f19abd91 on Nosferican:Update-Metadata-to-Julia-1 into 380d3d1428df3356f15cb1093b164eecbfc0ed66 on dmbates:master.

Codecov Report

Merging #151 into master will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff           @@
##           master     #151   +/-   ##
=======================================
  Coverage   89.57%   89.57%
=======================================
  Files          13       13
  Lines        1295     1295
=======================================
  Hits         1160     1160
  Misses        135      135

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 380d3d1...a6a763c. Read the comment docs.

Thanks again.

authors: Nosferican codecov-io coveralls dmbates


text:
HMww02 – Podcasting

In der ersten richtigen Folge erzählt Holger ein wenig aus dem Nähkästchen über seinen Podcast @autoweird.fm (Twitter). Wie kam es zum Podcast? Mit welcher Hardware nimmt er die Folgen auf? Welche Software kann er empfehlen und wie geht man eigentlich damit um, wenn im aktuellen Projekt Themen aufkommen, die auch im Podcast interessant sind?
Holger ist Senior IT Consultant bei codecentric, blogt dort nur über JavaScript und hatte auch irgendwann mal einen eigenen Blog. Für die Folge hat er sich den Laden Ein als Aufnahmeort ausgesucht.
@autoweird.fm startete mit Zencastr als Aufnahmesoftware und landete letztendlich bei Reaper und Ultraschall. "Herr Mies will's wissen" nutzt hier übrigens Ferrite und Podlove. Gehostet wird alles in der Cloud, da Soundcloud nicht voll überzeugen konnte. Holger und Benedikt erstellen mit Hilfe von Github Pages ihre Seite und Ihren Feed (bis das neue Layout eventuell den Feed killt).

Kapitelmarken fielen leider der Nachbearbeitung zum Opfer und der Pop-Filter ist bestellt. Außenaufnahmen finde ich persönlich von der Atmosphäre her ganz gut, gegen Ende war es dann aber doch ganz laut. Die nächsten beiden Episoden sind dann wahrscheinlich erstmal in einem Gebäude.

Die erste "richtige" Folge hat mir sehr gut gefallen. Ich fand die Hintergrundgeräusche nicht massiv störend, obwohl ich mit Kopfhörern gehört habe. Hat irgendwie zum Interview Flair beigetragen. Nur manchmal waren die Stimmen im Hintergrund relativ laut. Ich finde es gut, dass du dich auf deinen Gast vorbereitest und ihn vorstellst. Außerdem hat man gemerkt, dass du einen roten Faden hattest, auch wenn du ihm nicht komplett gefolgt bist.
Freue mich schon auf die nächsten Folgen. Weiter so! 👍

Vielen Dank. Ich hoffe die nächsten Folgen gefallen Dir ähnlich gut.

authors: britter dmies

repo: dmies/herr-mies-wills-wissen


text:
[BERT] Multi-GPU Training with Tree reduce

BERT_BASE and BERT_LARGE contains 110M and 340M parameters respectively. Currently multi-GPU scaling is poor for this model and the result shows large overhead for cross-GPU ndarray copies.
The default kvstore push/pull do not leverage the communication pattern on the machine (e.g. AWS p3 instance). It would be great to use the experimental tree reduce push/pull introduced by @ctcyang.
However, the following error occurs for MXNET_KVSTORE_LOGTREE=1 MXNET_KVSTORE_USETREE=1
[07:02:29] src/kvstore/./././gpu_topology.h:60: Weight:
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 2 2 0 0 0 0 0
[07:02:29] src/kvstore/./././gpu_topology.h:67: 2 0 0 2 0 0 0 0
[07:02:29] src/kvstore/./././gpu_topology.h:67: 2 0 0 0 0 0 2 0
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 2 0 0 0 0 0 2
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 0 0 0 0 2 2 0
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 0 0 0 2 0 0 2
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 0 2 0 2 0 0 0
[07:02:29] src/kvstore/./././gpu_topology.h:67: 0 0 0 2 0 2 0 0

[06:50:46] src/kvstore/././comm_tree.h:381: Using Kernighan-Lin to generate trees
[06:50:46] src/kvstore/./././gpu_topology.h:1030: No valid binary tree found from root 0, try backtracking
Traceback (most recent call last):
  File "run_pretraining.py", line 257, in <module>
    train()
  File "run_pretraining.py", line 228, in train
    trainer.step(1)
  File "/home/ubuntu/mxnet/python/mxnet/gluon/trainer.py", line 290, in step
    self._allreduce_grads()
  File "/home/ubuntu/mxnet/python/mxnet/gluon/trainer.py", line 320, in _allreduce_grads
    self._kvstore.push(i, param.list_grad(), priority=-i)
  File "/home/ubuntu/mxnet/python/mxnet/kvstore.py", line 237, in push
    self.handle, mx_uint(len(ckeys)), ckeys, cvals, ctypes.c_int(priority)))
  File "/home/ubuntu/mxnet/python/mxnet/base.py", line 252, in check_call
    raise MXNetError(py_str(_LIB.MXGetLastError()))
mxnet.base.MXNetError: [06:50:46] src/kvstore/./././gpu_topology.h:1040: No valid binary tree found from root 0 using backtracking

@eric-haibin-lin looks like this was resolved?

We have a fallback patch instead of a complete fix..

This seems to be better tracked in MXNet.

Any update on the status?

authors: eric-haibin-lin kaonashi-tyc szha

repo: dmlc/gluon-nlp


text:
[CI] Revert "[CI] Update CI dependencies (#756)"

Description
This reverts commit 0a0bb6c08563db9a9b6244ee8115fd2562fb7e47.
Checklist
Essentials

[x] PR's title starts with a category (e.g. [BUGFIX], [MODEL], [TUTORIAL], [FEATURE], [DOC], etc)
[x] Changes are complete (i.e. I finished coding on this PR)

Changes

[x] revert #756

Comments

After #756, the sphinx build runs into JSONDecodeError from time to time.

Job PR-769/1 is complete.
Docs are uploaded to http://gluon-nlp-staging.s3-accelerate.dualstack.amazonaws.com/PR-769/1/index.html

authors: mli szha


text:
Something Wrong with the fit and predict method

I am trying to do the fine tuning based on the Inception BN model.
But I find that even given the same data iterator, the training an evaluation accuracy differs a lot, while the evaluation accuracy is always 1/#class, which means the model produces replicate results for different data.
I also tried to modify
self._pred_exec.forward(is_train=False)

to
self._pred_exec.forward(is_train=True)

in model.py
It works for NDArrayIter but not ImageRecordArray

Looking into it. Possible bugs introduced in recent PRs

Try to modify the value of fix_gamma in BatchNorm from defaulted ‘’True‘’ to ''False'', since the default value is changed in recent commits.

authors: LiamZhuuu sodabeta7 winstywang

repo: dmlc/mxnet


text:
Bug if do backward on symbol mean

For bugs or installation issues, please provide the following information.
The more information you provide, the more likely people will be able to help you.
Environment info
Operating System: Ubuntu 14.04
Compiler: gcc4.8.4
Package used (Python/R/Scala/Julia): Python
MXNet version: Master Latest
Or if installed from source:
MXNet commit hash (git rev-parse HEAD):
If you are using python package, please provide
Python version and distribution: 2.7
If you are using R package, please provide
R sessionInfo():
Error Message:
Please paste the full error message, including stack trace.

Traceback (most recent call last):
File "test.py", line 20, in
mod.backward([mx.nd.array([1.0], dtype=np.float32)])
File "/home/lisabug/.virtualenvs/nnvm/local/lib/python2.7/site-packages/mxnet-0.9.4-py2.7-linux-x86_64.egg/mxnet/module/module.py", line 487, in backward
self._exec_group.backward(out_grads=out_grads)
File "/home/lisabug/.virtualenvs/nnvm/local/lib/python2.7/site-packages/mxnet-0.9.4-py2.7-linux-x86_64.egg/mxnet/module/executor_group.py", line 437, in backward
end=islice.stop)
File "mxnet/cython/ndarray.pyx", line 167, in ndarray._make_ndarray_function.generic_ndarray_function (mxnet/cython/ndarray.cpp:3602)
File "mxnet/cython/base.pyi", line 36, in ndarray.CALL (mxnet/cython/ndarray.cpp:1731)
mxnet.base.MXNetError: [16:15:17] src/operator/tensor/./matrix_op-inl.h:1028: Check failed: (*end <= axis_size) && (*end >= 0) Invalid begin, end, get begin=0, end=3

Minimum reproducible example
if you are using your own code, please provide a short script that reproduces the error.
import mxnet as mx
import numpy as np

data = mx.sym.Variable('data')
data = mx.sym.mean(data)

mod = mx.mod.Module(symbol=data, data_names=('data',), label_names=None)
mod.bind(data_shapes=[('data', (3, 1))])
mod.init_params()
mod.init_optimizer()

x = mx.nd.array([[1.0], [2.0], [3.0]], dtype=np.float32)

batch = mx.io.DataBatch([x], label=None)
mod.forward(batch, is_train=True)
mod.backward([mx.nd.array([1.0], dtype=np.float32)])
mod.update()

Steps to reproduce
or if you are running standard examples, please provide the commands you have run that lead to the error.

What have you tried to solve it?

@lisabug Due to the automatic slicing in the backward.
https://github.com/dmlc/mxnet/blob/master/python/mxnet/module/executor_group.py#L431-L443
The slice is determined by the batch_size, which is assumed to be 3 in this example.
https://github.com/dmlc/mxnet/blob/master/python/mxnet/module/executor_group.py#L210-L211

@sxjscience Yes. The automatic slicing itself caused the problem of backward on mean symbol since the output of mean symbol is a scalar instead of a tensor of size (b,).

Close it now and we suggest using gluon in the future.

authors: lisabug sxjscience


text:
DType support for BatchNorm

cc @Godricly
depends on https://github.com/dmlc/mshadow/pull/141

can you make it run using fp16? I tried on m40 but got cudnn fails. while the fp64 case is functional.

@vchuravy I just did some update on DTypeBN. However there are some issues to be fixed.

different mean and variance type in fp16 for cudnn and mshadow
fp16 mshadow BN seems not working

@Godricly I will look into it again later next week.

I won't have time to work on this, anytime soon.

authors: Godricly vchuravy


text:
fix issue 3645

Fix inconsistent checkpoint file name for saving and loading in example/image-classification/train_model.py.

I think the general idea was to avoid kv.rank in single machine training. Maybe consider only add it when number of kv nodes is bigger than 1? @mli

i think it is a good idea to add -%d % (kv.rank) only if kv.rank > 0

This issue is fixed by the latest commit by #3999

authors: mli tqchen zorksylar


text:
Merge into MXNet

@piiswrong
Any works/advice to do before we merge this into MXNet? I was going to write a script to auto-build the wheel file using Travis-CI and user could install TensorBoard simply using pip install tensorboard, just like TF does. Otherwise, the user has to install bazel and it makes MXNet's installation too heavy?

Have you figured out how to do this? Would be great if this works.

@piiswrong @szha
I followed the script as the one in mxnet-distro, but bazel build for tensorboard is too slow, https://travis-ci.org/zihaolucky/tensorboard/jobs/200914155#L3193-L3214
Any ideas? I found the build time for mac must be less than 50mins, or it would be terminated.

I've tried to change the bazel config(ram_utilization_factor or local_resources) for speedup, but didn't work.

It's compiling tensorflow ops. Can you make it only compile tensorboard? Try modifying bazel config file?

Good idea, I would try.
I compared the difference between building tensorboard and entire tensorflow when I first noticed this, the number of jobs is different, yet I supposed these are necessary but not trying to modifying the config file ><

I think it might work, as we've replaced the logging part, the tensorflow ops could be removed.

I used snakefood to analyze the dependencies of the rendering part:
sfood -fuq tensorflow/tensorboard/tensorboard.py | sfood-filter-stdlib | sfood-target-files > deps.log

And found the unnecessary tensorflow ops are imported from several files(tensorflow/tensorflow/python/summary/event_accumulator.py, etc), while the methods it uses doesn't require tensorflow ops so it's possible to replace them.  Another work is to change the bazel config/BUILD and make sure the SWIG files won't cause huge troubles.
However, it could make our project not as easy to maintain as we change the rendering part and the BUILD file, any ideas? Should we consider building the project using Jenkins like TF does? Ref:https://ci.tensorflow.org/view/Nightly/job/nightly-matrix-cpu/TF_BUILD_IS_OPT=OPT,TF_BUILD_IS_PIP=PIP,TF_BUILD_PYTHON_VERSION=PYTHON2,label=mac-slave/

If we use Jenkins, I have to setup a machine and envs to do that. But the TF has provided reusable scripts.

@piiswrong Could you help check https://travis-ci.org/zihaolucky/tensorboard/jobs/203088383#L645? It's okay in mac, but always failed in linux.
./configure < ../tools/pip_wheel/configure.conf

ill take a look later.
we can make a private repo for distribution so you don't need to hack compile time

Thanks! To hack the compile time limited in Travis, I have to remove some dependencies in its bazel config. I think it's okay if that won't cause any unexpected error/bug in the future, but technically it shouldn't happen as we only remove the unnecessary ops which tensorboard won't use.
So I would like to keep the Travis way currently(for simplicity) and focus on the logging part, as we have graph, video and even embedding to do.
The error in linux is caused by:
./configure < ../tools/pip_wheel/configure.conf

in which it has to configure some install options before bazel can start, I put the config in /tools/pip_wheel/configure.conf in the script as we have no keyboard input. My branch https://github.com/zihaolucky/tensorboard/tree/travis_pip_wheel

@piiswrong Now we have PyPI, how about add tensorboard as default requirements for MXNet Python package?
Any work I should do?

More docs about tensorboard.
Any changes in MXNet?

we can add some callbacks in mxnet.
Let's start with optional import and see if anyone have trouble installing

Yep.

Merged in mxnet.contrib.tensorboard

authors: piiswrong zihaolucky

repo: dmlc/tensorboard


text:
Can I add more features of data for training continuation?

As title, can I append some features to train a new model base on an old model?
For example, I trained a model with features [A, B, C, D, E] as an old model. After some months, I collect more data and features and want to update the model. I continue to train a new model with features [A, B, C, D, E, F] base on the old one.
I noticed I can pass the parameter xgb_model for training continuation, such as (python)
bst = xgb.train(params, d_train, num_round, watch_list, early_stopping_rounds=early_stopping, xgb_model=old_model_path)

The code can run, but I'm not sure that appending features is correct on algorithm.

Why would you do that? Learning on the whole dataset, including new data will give you the best model because it will use all data available from scratch.

@pommedeterresautee many causes. Maybe the whole dataset is too large or the data shift. Iteration with new data for adapting to the change of actual data.

Ok but you should start with all features.

@pommedeterresautee Aha! It makes sense! I got it! Thx!

authors: fcbruce pommedeterresautee

repo: dmlc/xgboost


text:
I want to use a set of features to correspond to multiple columns of lables

I want to use a set of features to correspond to multiple columns of lables

@acodercat We do not support multiple labels currently.


text:
Error when creating mapping large dataframe - started happening a few days ago.

Nothing has changed on my end so something must have been updated. I am using a AWS instance with 4x T4 GPUs.
All of the sudden I am getting this error when I try to create my DaskDeviceQuantileDMatrix:
from dask_cuda import LocalCUDACluster
from dask.distributed import Client
from dask import dataframe as dd
import xgboost as xgb
from xgboost.dask import DaskDMatrix
from dask.distributed import wait
import cupy as cp
import numpy as np
import pandas as pd

cluster = LocalCUDACluster()
client = Client(cluster)

xTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,220))),npartitions = 4)
yTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,1))),npartitions = 4)
xTrainG = xTrain.map_partitions(cp.array).persist()
yTrainG = yTrain.map_partitions(cp.array).persist()
wait([xTrainG,yTrainG])

dTrain = xgb.dask.DaskDeviceQuantileDMatrix(client=client, data=xTrainG, label=yTrainG, feature_names=features, max_bin=100)

Here is the error:
---------------------------------------------------------------------------
ValueError                                Traceback (most recent call last)
<ipython-input-10-b35d47d9db17> in <module>
      1 xTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,220))),npartitions = 4)
      2 yTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,1))),npartitions = 4)
----> 3 xTrainG = xTrain.map_partitions(cp.array).persist()
      4 yTrainG = yTrain.map_partitions(cp.array).persist()
      5 wait([xTrainG,yTrainG])

~/.conda/envs/python/lib/python3.7/site-packages/dask/base.py in persist(self, **kwargs)
    254         dask.base.persist
    255         """
--> 256         (result,) = persist(self, traverse=False, **kwargs)
    257         return result
    258

~/.conda/envs/python/lib/python3.7/site-packages/dask/base.py in persist(*args, **kwargs)
    756                 if client.get == schedule:
    757                     results = client.persist(
--> 758                         collections, optimize_graph=optimize_graph, **kwargs
    759                     )
    760                     return repack(results)

~/.conda/envs/python/lib/python3.7/site-packages/distributed/client.py in persist(self, collections, optimize_graph, workers, allow_other_workers, resources, retries, priority, fifo_timeout, actors, **kwargs)
   2950             retries=retries,
   2951             user_priority=priority,
-> 2952             fifo_timeout=fifo_timeout,
   2953             actors=actors,
   2954         )

~/.conda/envs/python/lib/python3.7/site-packages/distributed/client.py in _graph_to_futures(self, dsk, keys, workers, allow_other_workers, priority, user_priority, resources, retries, fifo_timeout, actors)
   2540             if not isinstance(dsk, HighLevelGraph):
   2541                 dsk = HighLevelGraph.from_collections(id(dsk), dsk, dependencies=())
-> 2542
   2543             dsk = highlevelgraph_pack(dsk, self, keyset)
   2544

~/.conda/envs/python/lib/python3.7/site-packages/dask/highlevelgraph.py in __dask_distributed_pack__(self, client, client_keys)
    962                 }
    963             )
--> 964         return dumps_msgpack({"layers": layers})
    965
    966     @staticmethod

~/.conda/envs/python/lib/python3.7/site-packages/distributed/protocol/core.py in dumps_msgpack(msg, compression)
    161                         coll = put_in(keys[:-1], coll, holder)
    162                     holder[keys[-1]] = val
--> 163                 else:
    164                     coll = val
    165                 return coll

~/.conda/envs/python/lib/python3.7/site-packages/msgpack/__init__.py in packb(o, **kwargs)
     33     See :class:`Packer` for options.
     34     """
---> 35     return Packer(**kwargs).pack(o)
     36
     37

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer.pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

msgpack/_packer.pyx in msgpack._cmsgpack.Packer._pack()

ValueError: bytes object is too large

 ValueError                                Traceback (most recent call last)
<ipython-input-10-b35d47d9db17> in <module>
      1 xTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,220))),npartitions = 4)
      2 yTrain = dd.from_pandas(pd.DataFrame(np.zeros(shape=(14000000,1))),npartitions = 4)
----> 3 xTrainG = xTrain.map_partitions(cp.array).persist()
      4 yTrainG = yTrain.map_partitions(cp.array).persist()
      5 wait([xTrainG,yTrainG])

That doesn't look like xgboost related?


text:
Failed to training with xgboost conda

/home/vi/baseline/lightgbm_xgboost.ipynb Cell 23' in <cell line: 2>()
1 xg_clf = xgb.XGBClassifier()
----> 2 xg_clf.fit(X_train,y_train)
File ~/miniconda3/envs/ds/lib/python3.8/site-packages/xgboost/core.py:532, in _deprecate_positional_args..inner_f(*args, **kwargs)
530 for k, arg in zip(sig.parameters, args):
531     kwargs[k] = arg
--> 532 return f(**kwargs)
File ~/miniconda3/envs/ds/lib/python3.8/site-packages/xgboost/sklearn.py:1382, in XGBClassifier.fit(self, X, y, sample_weight, base_margin, eval_set, eval_metric, early_stopping_rounds, verbose, xgb_model, sample_weight_eval_set, base_margin_eval_set, feature_weights, callbacks)
1377     params["num_class"] = self.n_classes_
1379 model, metric, params, early_stopping_rounds, callbacks = self._configure_fit(
1380     xgb_model, eval_metric, params, early_stopping_rounds, callbacks
1381 )
-> 1382 train_dmatrix, evals = _wrap_evaluation_matrices(
1383     missing=self.missing,
1384     X=X,
1385     y=y,
1386     group=None,
1387     qid=None,
1388     sample_weight=sample_weight,
1389     base_margin=base_margin,
1390     feature_weights=feature_weights,
...
[bt] (6) /home/vi/miniconda3/envs/ds/lib/python3.8/lib-dynload/_ctypes.cpython-38-x86_64-linux-gnu.so(+0x13c95) [0x7faa9551ac95]
[bt] (7) /home/vi/miniconda3/envs/ds/bin/python(_PyObject_MakeTpCall+0x3bf) [0x55f9eab4813f]
[bt] (8) /home/vi/miniconda3/envs/ds/bin/python(_PyEval_EvalFrameDefault+0x5434) [0x55f9eabf2dd4]

Could you please share the complete log and a reproducible example?

Closing due to lack of response.

authors: Shindora trivialfis


text:
"commit_message" is missing

Hi,
thank you for this package. It might be exactly what I need for package splitting.
I have a trouble with missing "commit_message":

Do I have a typo or something?

#12
I think my pull request should resolve this issue.

Hey @dmnemec, would it be possible to merge my pull request?

Closed by #12 , thanks. If you want to write a test for it to make sure it adds a message, that'd be cool too.

authors: TomasVotruba bboles dmnemec


text:
Using undocumented externalDependencies instead of peerDependencies potentially causing issues

Environment

Tech
Version

material-ui-pickers
1.0.0-rc.5

material-ui
1.0.0-beta.40

React
16.3.2

Browser
Chrome

Platform
Windows 10

Steps to reproduce

Use yarn workspaces and install the required peer dependency of date-fns@^2.0.0-alpha.7 inside my /workspace/vs-frontend package.
Also rely on other packages which require date-fns, but an earlier incompatible version (webpack-cli for example).
Because yarn uses hoisting, it is hoisting material-ui-pickers up to a root node_modules folder, and the other date-fns packages, but leaving our specific version of date-fns@^2.0.0-alpha.7 behind because there is no known link between material-ui-pickers and that specific package and version.

Expected behavior

Busy getting feedback about the issue specifically relating to material-ui-pickers here: https://github.com/yarnpkg/yarn/issues/5705
Basically the expected behaviour would be that these two modules remain aligned in the same node_modules level.
Actual behavior

Instead I now have this folder structure:
node_modules
  |- date-fns@1.29.0
  |- vs-worker
  |    +- node_modules
  |        +- date-fns@2.0.0-alpha.7
  |- vs-frontend
  |    +- node_modules
  |        +- date-fns@2.0.0-alpha.7
  |- concurrently
  |- material-ui-pickers
  +- listr-verbose-renderer

Which is problematic for material-ui-pickers resolving the correct date-fns version. And I get this error:
Error: Cannot find module 'date-fns/addDays'
    at Function.Module._resolveFilename (internal/modules/cjs/loader.js:548:15)
    ...

I'm busy getting feedback, but it seems that perhaps setting this external dependency correctly as a peerDependency might fix it. externalDependencies is not documented anywhere in the NPM spec, so I'm unsure why it is being used here.

About different versions of date-fns - I cant imagine fast solution, because if libs are relying on different versions of the same library - how the peerDependencies will fix your problem

That's what I'm currently following up on here: https://github.com/yarnpkg/yarn/issues/5705
The general idea is that peerDependencies make an "official" link between material-ui-pickers and date-fns@^2.0.0-alpha.7. This would help package mangers know to place material-ui-pickers in a node_modules context in which it can access the correct version of date-fns.
The libs that require the different versions will generally then have their required version of date-fns as a sub-module, as is usual, such as: node_modules/web-cli/node_modules/date-fns because yarn or npm sees the clash. Currently because you are using an undocumented externalDependencies, we can't expect yarn or npm to ever recognise that relationship and see the problem.

Actually no. Because of for peer dependencies there is no ability to select one of date-fns, moment or luxon. Thus anytime date-fns users will have an error of not installed moment and luxon - thats would be an obvious and might be a reason of overheading installation.

Yea, I understand why this is an issue. This has been an issue for many when it comes to npm and peerDependencies - as seeing those warning messages for packages that you don't need installed is annoying.
But until npm has given us a better option, I think putting up with the warnings might be the best option. PeerDependencies are not installed automatically as far as I know, so it shouldn't cause overhead - only annoying warning messages.

What about publishing three versions of the library to npm, with the correct peerDependency (or actual dependency)?

@Haroenv thats should be a lot of libs to manage) I think would be more efficient to let npm write error messages.

@Haroenv it's harder to maintain three versions :/
What if we remove externalDependencies?
We still have info about peer dependencies in Getting Started docs, and there's message in console after library is installed. It should be enough.
Also, what benefits has externalDependencies in package.json? If it just causes errors with yarn, it should be safe to remove it ;)

@cherniavskii externalDependencies its a custom section its not worked with yarn anyway. Yarn needs peerDependencies to resolve same library versions (date-fns v1 & v2) - and if some another lib relying on date-fns 1 its not working with material-ui-pickers.

@lostpebble what if you will say that you need date-fns 2 actually npm i date-fns@next --save

externalDependencies its a custom section its not worked with yarn anyway

yeah, that's why I suggest to remove it ;)

@cherniavskii thats just a place to display that lib using some another library, because we are just ignoring moment, date-fns and luxon somewho will maybe looking for information about these libs in the package.json of installed package in node_modules.

@dmtrKovalenko that's what I've done, I've officially included it in my package.json as a direct dependency. But because of the way yarn hoisting works in workspaces, and because I have other modules which have the date-fns@1.x library requirement (in their own package.json), it not seeing material-ui-pickers as requiring this specific version of date-fns@next, so it doesn't "hoist" them together and keep them in the same /node_modules context.
You can see in great detail what I'm talking about, as we've been discussing this exact problem with the yarn team over here: https://github.com/yarnpkg/yarn/issues/5705#issuecomment-383388063
In a regular project it wouldn't cause any issues, but in monorepos it is causing issues because of the way their algorithms work to preserve space. They would only ensure material-ui-pickers and date-fns@next will stay together if there is an officially defined link between them (peerDependencies).
So yea @cherniavskii , its not so much the issue of externalDependencies being there - but rather that peerDependencies is not being used instead.

I can imagine a build process which simply edits the package.json in a new folder for each of the versions and publishes all three with the same code but different package.json#peerDependencies to three packages. That would be a lot simpler. In a later stage I guess you could put the code that handles with the different dependencies in separate files too, which would lower the overhead of it (if that's necessary / makes sense)

I am actually aware of 3 packages - due to overhead. Using this lib should be as easy as possible. I will think about peerDependencies and maybe will include all libs to peer in the next release (rc.8)
But I think we will not integrate them to peerDependencies. Only one workaround I can imagine - create your own utils and use them. But a lot of errors for each build - not really good solution.

Another option would be to use the standard optionalDependencies field.

@RedHatter thanks for sharing this info here, didn't know that optionalDependencies is standardized field.
@dmtrKovalenko I think we should use that, instead of externalDependencies :)

Moved to optional dependencies

authors: Haroenv RedHatter cherniavskii dmtrKovalenko lostpebble

repo: dmtrKovalenko/material-ui-pickers


text:
TotalInputLumis TotalInputEvents TotalInputFiles  are 0

https://cmsweb.cern.ch/reqmgr2/data/request?name=pdmvserv_task_JME-RunIISummer16DR80Premix-00002__v1_T_161115_223647_4976
made unified crash. Added a protection for such "feature"

duplicate #5486

authors: ticoann vlimant

repo: dmwm/WMCore


text:
Use port 8443 to communicate with cmsweb services

Fixes #8565
Replaces #8704
Instead of changing every single place that defines a cmsweb endpoint, we can simply change it in this base class and make sure the other Services modules inherit from it (there seems to be a few that don't).

smart. :smile:, Is there any case we are not using Service class?  how about dbs apis?

Yes, I see quite a few, including DBS3Reader and possibly some legacy modules.
I'll create those changes in another PR, if it's straight forward, then we add to today's agent release, otherwise this change will touch the bulk of the cmsweb requests.

BTW, I ran the DMWM tests on an agent with this patch and I see data was uploaded to the DQM GUI, workflows properly completed, dbs and phedex stuff also done. I think we're good with this change.

Thanks Alan,

Before I forget, @belforte @hufnagel if you build T0/CRAB using tag 1.1.15 or newer, this change will be in. So any of those modules under WMCore/Services that inherit from Service class, will be talking to CMSWEB services through the port 8443 instead of 443.
It should be transparent to your services, but just in case :)

thanks

Hi Alan, I am likely missing something, but looking at what's new there (to get a sense
of what to test) I do not see anything in the way of DBS URL.
On 10/08/18 20:57, Alan Malta Rodrigues wrote:

Before I forget, @belforte https://github.com/belforte @hufnagel https://github.com/hufnagel if you build T0/CRAB using tag 1.1.15 or newer, this change will be in. So any of those modules under WMCore/Services that inherit from |Service| class, will be talking to CMSWEB services through the port 8443 instead of 443.
It should be transparent to your services, but just in case :)

forgot the URL I have used to compare
https://github.com/dmwm/WMCore/compare/1.1.14_crab...1.1.15_cmsweb
On 17/08/18 18:30, Stefano Belforte wrote:

Hi Alan, I am likely missing something, but looking at what's new there (to get a sense
of what to test) I do not see anything in the way of DBS URL.
On 10/08/18 20:57, Alan Malta Rodrigues wrote:

Before I forget, @belforte https://github.com/belforte @hufnagel https://github.com/hufnagel if you build T0/CRAB using tag 1.1.15 or newer, this change will be in. So any of those modules under WMCore/Services that inherit from |Service| class, will be talking to CMSWEB services through the port 8443 instead of 443.
It should be transparent to your services, but just in case :)

Hi Stefano, you are right. DBS is not included here. We can temporary switch DBS port here by overwriting URLs in WMCore/Service/DBS package, Or wait for DBS client source handle this. I think it is cleaner dbs client handle that. But if you prefer we can add that here temporary.

thanks @ticoann. Let's wait. I do not need do to test things now. Only wanted to have a sense of what to look if I build with new tag.

authors: amaltaro belforte ticoann


text:
List of WMAgent python dependencies

Initial version of the packages required by WMAgent, extracted from the investigation carried by Erik on making WMAgent package available through Pypi.
I'm creating it now because I'd like to have the "Dependency Graph" enabled and start getting alerts from Github on security vulnerabilities found in WMCore (python) dependencies.
Most of the versions in this requirements list are from our CMS repository. Only a few of them were updated on Erik's environment, thus I expect to receive a few alarms asking to upgrade dependencies as soon as it gets merged.

Great idea!

Thanks Alan

authors: amaltaro bbockelm ticoann


text:
Scale up and scale back tier0-reaper pods

Temporarily scaling T0 reaper pods (which work on T0_CH_CERN_Disk and T2_CH_CERN .
We have a mismatch in deletion rate: https://monit-grafana.cern.ch/d/WPDxqoL4z/rucioaccounts?orgId=11&var-RSE=T2_CH_CERN&var-AccountType=All&from=now-2d&to=now&viewPanel=9&var-AccountName=All&var-ArchivalAccount=All&var-diskserviceaccount=All
I suspect this is because of the slow response of ddm_quota changes.
Will move this operation to the hourly cron job.

Reverted here, https://github.com/dmwm/rucio-flux/pull/195

repo: dmwm/rucio-flux


text:
add ListingHelper to access iterator information like $Pos, $First, $Last and $EvenOdd

Hi there,
We had been developing a similar module (mainly for SS3) where we had the necessity to access the iterator information, like $Pos, $First, $Last and $EvenOdd when iterating over blocks. This is a simple helper to achieve just this. Let me know if you need anything or you had a different approach in mind.

Hi there @banglashi, I'm sorry we didn't get to review this sooner! It looks like #282 now adds the same kinds of features. Would you mind taking a look and seeing whether you think this pull request could still add anything over that? If not, I'll close this one

Hi @robbieaverill
no problem for the delay. The merge request looks like it fulfils the same features so I will close this one. Thanks.

authors: banglashi robbieaverill

repo: dnadesign/silverstripe-elemental


text:
Job abstraction

These are the current dobi's resources:
// Build tool specific
- env
- alias
// You are nice
- compose
// What the end user actually run/expects
- image
- job
// you cant use a mount on a image. Its strictly job related
- mount

But images are a boring abstraction. Compose deals with them super nice already. You just run from a file or ou pull it from the hub. Its how you actually run them is what makes dobi special. We should make the job abstraction stronger for itself. I think that ultimately dobi should support a single manifest abstraction for running a job, in the same sense of #55, alowing the end user to have a single file with all the information about the job.
job=watch:
    image:
        // I dont like this. currently `image=refactorer:    image: dnephin/go-refactor` is a stutterer
        // I would rather container, container-name or repo or name. IDK
        container: dnephin/dobi-dev
        steps:
          - FROM: golang:1.7-alpine
          - RUN: apk add -U git bash curl tree
          - RUN: go get github.com/dnephin/filewatcher
          - GLIDE: 0.12.3
    command: filewatcher x '.git' -- go run main.go
    interactive: true
    working-dir: /go/src/github.com/dnephin/dobi
    description: "Watch for code changes and run"
    env:
      - 'PS1=# '
      - 'CGO_ENABLED=0'
    local-mounts:
        - bind: .
          path: /go/src/github.com/dnephin/dobi

The main intent here is to make dobi more concise, avoiding the file changing dizzyness as well as saving some lines from the dockerfile itself, once that we dont need to set things like WORKDIR and ENV because we enforced them on the way the job run. So when you are actually hands on writing a dobi resource, you either decide if you need a image or a job. You might use the other stuff, but only if you want/need. You would also be able to keep a 1:1 ratio of the tasks you will actually be able to run with the number of resource you wrote.
To do that we need to

Implement local-mount field on a job, seem easy.
Implement a image field tha would embed a image type inside a job to run instead of use field, which sounds way trickier.

Sorry, but I don't like this. I consider the current design one of the strong points of dobi. I should probably document why it is designed this way.
There are a lot of disadvantages to "inlined" resources:

it makes it impossible to manage the resources individually. job:pull doesn't make any sense. It needs to be image:pull.
it is a lot less clear to someone looking at the file which properties are for which part of the resource
you can't re-use the resource, which means you could accidentally add duplication
it adds bloat to the format, because there would be multiple ways to define the same thing, instead of a single way

Where as the only disadvantage to having them separated is that there are a few extra lines in the file.
I think it's also important to remember that dobi is still very new. There will be more resources added (for things like services, deployments, networks, etc). If we go mixing everything into one it's going to be really messy trying to add new resource types. The existing resources could also expand.mount for example is likely to grow to include named volumes.

@dnephin you are right. I just realized this now that I was going through how tasks are created.
I still do not see any major issue with with adding localmounts to a job though

authors: cescoferraro dnephin

repo: dnephin/dobi


text:
Simple hardware based configuration

Generally, for a given system (mostly based around GPU or system architecture and RAM), there will be an optimal LLM size.and context.
We should probably have settings for Mac's based on RAM and for x86 machines based on VRAM that give optimal performance.  This would avoid a ton of questions from non-technical people on which settings to use.

Does someone want to write a system analysis tool that checks for the presence and VRAM on the system?

repo: dnhkng/GlaDOS


text:
Humans divinde iki scroll barın bulunması

chrome;

firefox;

Çözüm;
En dıştaki scroll barı kaldırmak için;
components/SidebarHumans.vue dosyası 6. satırın
   .xl-ba-white.xl-fs14.xl-lh20.xl-br8.xl-bo-gray-200.xl-bw2.xl-os
sonundaki .xl-os classının çıkarılması
   .xl-ba-white.xl-fs14.xl-lh20.xl-br8.xl-bo-gray-200.xl-bw2
components/SidebarHumans.vue dosyası 58. satırdaki .humans classına aşağıdaki özelliğin eklenmesi;
eski;
 .humans {     max-height: 284px;     overflow: scroll;   }
eklendikten sonraki hali;
 .humans {     max-height: 284px;     overflow: scroll;     overflow-x: hidden;   }
Düzeltilmiş hali;
chrome;

firefox;

teşekkür ederim.

bu durum düzeltildi.

authors: dnomak tgrlgnc


text:
DVD Cannot playback in VLC

Thanks again for crouton. I've attached an external DVD drive to watch movies which identifies itself as
Bus 001 Device 002: ID 059f:0643 LaCie, Ltd

...and which I've proven spins up and can decrypt DVDs when attached to my Lubuntu Vivid Vervet machine. However, the same drive with the same disc fails to read when running on Crouton and VLC reports...
libdvdnav: Using dvdnav version 5.0.3
libdvdread: Attempting to use device /dev/sr0 mounted on /var/host/media/removable/PROPOSITION_DVD for CSS authentication
libdvdread: Could not open /dev/sr0 with libdvdcss.
libdvdread: Can't open /dev/sr0 for reading
libdvdread: Device /dev/sr0 inaccessible, CSS authentication not available.
libdvdnav: Can't read name block. Probably not a DVD-ROM device.
libdvdnav: vm: dvd_read_name failed
libdvdnav: DVD disk reports itself with Region mask 0x00fd0000. Regions: 2
[00007f2f40005db8] core input error: ES_OUT_RESET_PCR called
[00007f2f40005db8] core input error: ES_OUT_RESET_PCR called
[00007f2f40005db8] core input error: ES_OUT_RESET_PCR called
[00007f2f40005db8] core input error: ES_OUT_RESET_PCR called
libdvdnav: Using dvdnav version 5.0.3
libdvdread: Could not open /dev/sr0 with libdvdcss.
libdvdread: Can't open /dev/sr0 for reading
libdvdnav: vm: failed to open/read the DVD
libdvdread: Could not open /dev/sr0 with libdvdcss.
libdvdread: Can't open /dev/sr0 for reading
[00007f2f4800ae98] dvdread demux error: DVDRead cannot open source: /dev/sr0
[00007f2f40006198] core input error: open of `dvd:///dev/sr0' failed

I have installed libdvdcss2 within my LXDE crouton build by running the standard libdvdread4 install-css script exactly as I did on Vivid Vervet. I've added my user account to the disk group in the hope that would unravel permissions problems and rebooted, but still no joy.
It is a promising test case for attaching an external DVD drive as the identical USB setup on a regular Lubuntu install works fine.

Here is the contrasting case in Vivid Vervet with the same device and disc...
libdvdnav: Using dvdnav version 5.0.1
libdvdread: Attempting to use device /dev/sr1 mounted on /media/cefn/PROPOSITION_DVD for CSS authentication
libdvdnav: DVD disk reports itself with Region mask 0x00fd0000. Regions: 2
libdvdread: Attempting to retrieve all CSS keys
libdvdread: This can take a _long_ time, please be patient
libdvdread: Get key for /VIDEO_TS/VIDEO_TS.VOB at 0x00000138
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_01_0.VOB at 0x0000019c
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_01_1.VOB at 0x0000027b
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_02_0.VOB at 0x00026a13
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_02_1.VOB at 0x00026a60
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_03_0.VOB at 0x000286ca
libdvdread: Elapsed time 0
libdvdread: Get key for /VIDEO_TS/VTS_03_1.VOB at 0x00028717
libdvdread: Elapsed time 0
libdvdread: Found 3 VTS's
libdvdread: Elapsed time 0

...although I doubt there are any significant clues there.

I see this on my chroot.
$ ls -l /dev/sr0
brw-rw---- 1 root cdrom 11, 0 Jun 30 14:27 /dev/sr0

So, try this in your chroot
$ sudo usermod -a -G cdrom your_user_name

and log in to crouton again.
You should be able to play DVDs with this.

Hey thanks for the suggestion and it should have been obvious if I'd actually inspected the device node under /dev
I succeeded at playing a DVD by adding myself to the cdrom group, then forcing a reload of my group memberships and launching VLC in a terminal as follows...
sudo usermod -a -G cdrom myuser
exec su -l myuser
vlc

...where myuser is my user name. After that, I removed and reinserted the DVD. My LXDE desktop session prompted for password and asked how to load the DVD. I chose VLC and it used the already-running instance with the renegotiated permissions and played successfully, without even a reboot. Obviously after a reboot the group membership of cdrom will be there without fancy terminal stuff.
I'll raise the issue of how to auto-mount drives without a password popup every time as a separate question.

play DVD crouton - Ubuntu 14-04
sudo apt-get update
sudo apt-get install vlc
run install libdvd -- https://github.com/dnschneid/crouton/issues/1367
sudo apt-get install libdvdread4
sudo /usr/share/doc/libdvdread4/install-css.sh
give permission to your user to play DVD -- @cefn
sudo usermod -a -G cdrom myuser
reload your user group
exec su -l myuser
run VLC
vlc

authors: alangumer cefn tmagara

repo: dnschneid/crouton


text:
Kali will not start xserver

Please paste the output of the following command here: sudo edit-chroot -all
name: kali-rolling
encrypted: no
Entering /mnt/stateful_partition/crouton/chroots/kali-rolling...
crouton: version 1-20161129162558~master:5f2f11f9
release: kali-rolling
architecture: amd64
xmethod: xorg
targets: xfce-desktop,extension
host: version 8872.73.0 (Official Build) stable-channel terra
kernel: Linux localhost 3.18.0-13436-g8b496ef #1 SMP PREEMPT Sat Dec 17 11:05:03 PST 2016 x86_64 GNU/Linux
freon: yes
Unmounting /mnt/stateful_partition/crouton/chroots/kali-rolling...

Please describe your issue:
I'm trying to start XFCE with the startxfce4 command, however when I do, it fails even with a fresh chroot.
I get this as the output:
/usr/bin/startxfce4: Starting X server
_XSERVTransmkdir: Owner of /tmp/.X11-unix should be set to root
X.Org X Server 1.19.0
Release Date: 2016-11-15
X Protocol Version 11, Revision 0
Build Operating System: Linux 3.16.0-4-amd64 x86_64 Debian
Current Operating System: Linux localhost 3.18.0-13436-g8b496ef #1 SMP PREEMPT Sat Dec 17 11:05:03 PST 2016 x86_64
Kernel command line: cros_secure console= loglevel=7 init=/sbin/init cros_secure oops=panic panic=-1 root=/dev/dm-0 rootwait ro dm_verity.error_behavior=3 dm_verity.max_bios=-1 dm_verity.dev_wait=1 dm="1 vroot none ro 1,0 3584000 verity payload=PARTUUID=66c55ce6-bb17-c445-9309-ce315284b589/PARTNROFF=1 hashtree=PARTUUID=66c55ce6-bb17-c445-9309-ce315284b589/PARTNROFF=1 hashstart=3584000 alg=sha1 root_hexdigest=bdd2412288f8336bfcac1dd268e5fab6f0504c48 salt=a1578c2ad900eba4f4231ef0bffac23acff06c2da7478912286447eb47c0714d" noinitrd vt.global_cursor_default=0 kern_guid=66c55ce6-bb17-c445-9309-ce315284b589 add_efi_memmap boot=local noresume noswap i915.modeset=1 tpm_tis.force=1 tpm_tis.interrupts=0 nmi_watchdog=panic,lapic
Build Date: 16 December 2016  07:30:27PM
xorg-server 2:1.19.0-3 (https://www.debian.org/support)
Current version of pixman: 0.34.0
Before reporting problems, check http://wiki.x.org
to make sure that you have the latest version.
Markers: (--) probed, (**) from config file, (==) default setting,
(++) from command line, (!!) notice, (II) informational,
(WW) warning, (EE) error, (NI) not implemented, (??) unknown.
(++) Log file: "/tmp/Xorg.crouton.1.log", Time: Mon Jan  2 15:17:05 2017
(==) Using system config directory "/usr/share/X11/xorg.conf.d"
method return time=1483388225.391621 sender=:1.11 -> destination=:1.64 serial=424 reply_serial=2
boolean true
xf86EnableIOPorts: failed to set IOPL for I/O (Operation not permitted)
crouton: version 1-20161129162558~master:5f2f11f9
release: kali-rolling
architecture: amd64
xmethod: xorg
targets: xfce-desktop,extension
host: version 8872.73.0 (Official Build) stable-channel terra
kernel: Linux localhost 3.18.0-13436-g8b496ef #1 SMP PREEMPT Sat Dec 17 11:05:03 PST 2016 x86_64 GNU/Linux
freon: yes
_IceTransmkdir: Owner of /tmp/.ICE-unix should be set to root
xfce4-session: No SSH authentication agent found
gpg-agent[1416]: WARNING: "--write-env-file" is an obsolete option - it has no effect
gpg-agent[1416]: directory '/home/will/.gnupg' created
gpg-agent[1416]: directory '/home/will/.gnupg/private-keys-v1.d' created
gpg-agent[1417]: gpg-agent (GnuPG) 2.1.16 started
(xfce4-session:1399): xfce4-session-WARNING **: gpg-agent returned no PID in the variables
(xfce4-session:1399): xfce4-session-WARNING **: xfsm_manager_load_session: Something wrong with /home/will/.cache/sessions/xfce4-session-localhost:1, Does it exist? Permissions issue?
(xfwm4:1419): GLib-CRITICAL **: g_str_has_prefix: assertion 'prefix != NULL' failed
/usr/bin/xbindkeys_autostart: line 24: CONF: unbound variable
(xfwm4:1419): xfwm4-WARNING **: The property '/general/double_click_distance' of type int is not supported
process 1434: arguments to dbus_message_new_method_call() were incorrect, assertion "path != NULL" failed in file ../../../dbus/dbus-message.c line 1367.
This is normally a bug in some application using the D-Bus library.
(xfsettingsd:1428): GLib-CRITICAL **: g_str_has_prefix: assertion 'prefix != NULL' failed
(xfsettingsd:1428): GLib-GObject-CRITICAL **: g_value_get_string: assertion 'G_VALUE_HOLDS_STRING (value)' failed
(xfsettingsd:1428): GLib-GObject-CRITICAL **: g_value_get_string: assertion 'G_VALUE_HOLDS_STRING (value)' failed
/usr/share/system-config-printer/applet.py:45: PyGIWarning: Notify was imported without specifying a version first. Use gi.require_version('Notify', '0.7') before import to ensure that the right version gets loaded.
from gi.repository import Notify
system-config-printer-applet: failed to start NewPrinterNotification service
system-config-printer-applet: failed to start PrinterDriversInstaller service: org.freedesktop.DBus.Error.AccessDenied: Connection ":1.40" is not allowed to own the service "com.redhat.PrinterDriversInstaller" due to security policies in the configuration file
(xfwm4:1419): xfwm4-WARNING **: Another compositing manager is running on screen 0
Cannot open /sys/class/tty/tty0/active: No such file or directory
Error: target STRING not available
(II) AIGLX: Suspending AIGLX clients for VT switch
method return time=1483388256.578113 sender=:1.11 -> destination=:1.66 serial=425 reply_serial=2
boolean true
Error: target STRING not available
^[[23;2~method return time=1483388287.168628 sender=:1.11 -> destination=:1.69 serial=434 reply_serial=2
boolean true
(EE)
Fatal server error:
(EE) EnterVT failed for screen 0
(EE)
(EE)
Please consult the The X.Org Foundation support
at http://wiki.x.org
for help.
(EE) Please also check the log file at "/tmp/Xorg.crouton.1.log" for additional information.
(EE)
(II) AIGLX: Suspending AIGLX clients for VT switch
method return time=1483388287.199632 sender=:1.11 -> destination=:1.70 serial=435 reply_serial=2
boolean true
(EE) Server terminated with error (1). Closing log file.
xfce4-session: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.
(xfsettingsd:1428): libxfce4ui-WARNING **: ICE I/O Error
(xfsettingsd:1428): libxfce4ui-WARNING **: Disconnected from session manager.
Running exit commands...
/usr/bin/xinit: connection to X server lost
waiting for X server to shut down xfsettingsd: Fatal IO error 11 (Resource temporarily unavailable) on X server :1.0.
xfdesktop: Fatal IO error 4 (Interrupted system call) on X server :1.0.
Hangup
Hangup
I've been able to cause it to crash so that I can use the CLI, but I have to spam CTRL-ALT-SHIFT-BACK/FORWARD.
If known, describe the steps to reproduce the issue:
Start Kali-rolling, extension, and xfce-desktop in chroot.  Install kali-desktop-xfce package, remove xserver-legacy (or whatever it's called).  Put Chromebook to sleep, reopen.  You should need to force a shutdown, start up again, and you should get this error.

@pswiifanwmi This may or may not help but have a look at some wisdom here:
https://github.com/dnschneid/crouton/issues/2978#issuecomment-269849365

Update should fix this.

authors: drinkcat octopusnz pswiifanwmi


text:
⚠ Site ANTS has degraded performance

In c4c3d81, Site ANTS (https://ants.gouv.fr) experienced degraded performance:

HTTP code: 200
Response time: 5756 ms

Resolved: Site ANTS performance has improved in 2cb887c after 42 minutes.

repo: dnum-mi/stats-sites-api


text:
⚠ C3 API Personnes décédées has degraded performance

In dd11efb, C3 API Personnes décédées (https://deces.matchid.io/deces/api/v1/search?q=pompidou georges&fuzzy=false) experienced degraded performance:

HTTP code: 200
Response time: 4067 ms

Resolved: C3 API Personnes décédées performance has improved in bbed998 after 1 hour, 1 minute.


text:
🛑 Site telepoint is down

In 750d9a9, Site telepoint (https://tele7.interieur.gouv.fr) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Site telepoint is back up in 30c15fe.


text:
strange delay in transmitting after 60-70 bytes of data

I'm using (abusing) SPI to send data to a WS2812 LED string.
The code (got it from: https://github.com/joosteto/ws2812-spi):
def write2812(spi, data):
    tx=[0x00]
    for rgb in data:
        for byte in rgb:
            for ibit in range(3,-1,-1):
                tx.append(((byte>>(2*ibit+1))&1)*0x60 +
                          ((byte>>(2*ibit+0))&1)*0x06 +
                          0x88)
    spi.xfer2(tx,3200000)

Since the ws2812 only uses one wire for data & clock timing is crucial.
This works perfectly for 5 or fewer pixels, above that the behaviour becomes erratic.
So I connected a scope to the SPI (MOSI) Pin and discovered that after about 60-70 bytes the SPI stops for a brief time (about 5µs).
sceenshot
I don't think this is a buffer issue.
Could this be an issue with the hardware SPI? or maybe the py-spidev?
Oh, I'm using a H2+ CPU on a orange pi zero

Here's a possible solution: https://github.com/joosteto/ws2812-spi/issues/6#issuecomment-382293838

authors: melazarus pietrodn

repo: doceme/py-spidev


text:
Add procps to :scm variants since it's fairly common in build systems

This needs to be explicit since stretch has managed to remove "procps" from minbase again! :tada:

Before:
$ docker images
REPOSITORY      TAG       IMAGE ID      CREATED             VIRTUAL SIZE
buildpack-deps  sid       4726cff60aca  11 seconds ago      993 MB
buildpack-deps  sid-scm   da97298f15a9  About a minute ago  292.9 MB
buildpack-deps  sid-curl  3153f2460d5e  2 minutes ago       165.4 MB
debian          sid       4a3511211916  17 hours ago        116.7 MB

After:
$ docker images
REPOSITORY      TAG       IMAGE ID      CREATED             VIRTUAL SIZE
buildpack-deps  sid       46605af50a61  10 seconds ago      884.6 MB
buildpack-deps  sid-scm   e2170ab8e3e8  About a minute ago  293.6 MB
buildpack-deps  sid-curl  e7361b0fba68  About a minute ago  165.4 MB
debian          sid       4a3511211916  4 days ago          116.7 MB

LGTM

authors: tianon yosifkit

repo: docker-library/buildpack-deps


text:
Extend mongo image with ssh

I want to extend the mongo image with a ssh server
I’ve written this lines :
FROM mongo

RUN apt-get update && \
    apt-get install -y openssh-server

EXPOSE 22

RUN useradd -s /bin/bash -p $(openssl passwd -1 test) -d /home/nf2/ -m -G sudo test

CMD ["sh", "-c", "service ssh start", "bash"]

But this will run either ssh or mongo not both at the same time !
Any idea on how to to it ?
Thanks in advance

You're going to need some kind of process supervisor to run both processes.  Instead of running SSH inside your container, I'd suggest using SSH on the host, and using docker exec to get a shell inside the container.
In the future (and for further assistance), these sorts of questions/requests would be more appropriately posted to the Docker Community Forums, the Docker Community Slack, or Stack Overflow.

authors: mcherb tianon

repo: docker-library/mongo


text:
pgadmin connection error

On connection by pgadmin to new database it get error: Error connctiong to the server: server closed the connection unexpectedly This probably means the server terminated abnormally before or while processing the request

Could you attach the logs from your pgadmin and postgres containers? docker logs [id | name]

postgres logs:
2015-08-02 14:44:16 MSK LOG:  database system was shut down at 2015-08-02 02:16:41 MSK
2015-08-02 14:44:16 MSK LOG:  MultiXact member wraparound protections are now enabled
2015-08-02 14:44:16 MSK LOG:  autovacuum launcher started
2015-08-02 14:44:16 MSK LOG:  database system is ready to accept connections
2015-08-02 14:44:16 MSK LOG:  incomplete startup packet
pgadmin:
2015-08-02 14:46:47 STATUS : Соединение с базой данных...
2015-08-02 14:46:47 INFO   : Opening connection with connection string: host='localhost' dbname='postgres' user='postgres' password='XXXXXX' port=5432 application_name='pgAdmin III - Браузер'
2015-08-02 14:46:49 STATUS : Соединение с базой данных... (2,02 секунд)
2015-08-02 14:46:49 ERROR  : Error connecting to the server: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
docker logs:
ivan@ivan-VirtualBox:/var/log/postgresql$ docker logs pg
The files belonging to this database system will be owned by user "postgres".
This user must also own the server process.
The database cluster will be initialized with locale "en_US.utf8".
The default database encoding has accordingly been set to "UTF8".
The default text search configuration will be set to "english".
Data page checksums are disabled.
fixing permissions on existing directory /var/lib/postgresql/data ... ok
creating subdirectories ... ok
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting dynamic shared memory implementation ... posix
creating configuration files ... ok
creating template1 database in /var/lib/postgresql/data/base/1 ... ok
initializing pg_authid ... ok
initializing dependencies ... ok
creating system views ... ok
loading system objects' descriptions ... ok
creating collations ... ok
creating conversions ... ok
creating dictionaries ... ok
setting privileges on built-in objects ... ok
creating information schema ... ok
loading PL/pgSQL server-side language ... ok
vacuuming database template1 ... ok
copying template1 to template0 ... ok
copying template1 to postgres ... ok
syncing data to disk ... ok
WARNING: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
postgres -D /var/lib/postgresql/data

or
pg_ctl -D /var/lib/postgresql/data -l logfile start
FATAL:  the database system is starting up
LOG:  database system was shut down at 2015-08-02 11:45:57 UTC
PostgreSQL init process in progress...
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started
ALTER ROLE
/docker-entrypoint.sh: ignoring /docker-entrypoint-initdb.d/*
LOG:  received smart shutdown request
LOG:  autovacuum launcher shutting down
LOG:  shutting down
LOG:  database system is shut down
PostgreSQL init process complete; ready for start up.
LOG:  database system was shut down at 2015-08-02 11:45:59 UTC
LOG:  MultiXact member wraparound protections are now enabled
LOG:  database system is ready to accept connections
LOG:  autovacuum launcher started

It looks like your pgadmin is trying to connect to localhost, if that is being run in a container you will need to tell it to connect to the IP address of the postgres container. Here is how I connect with psql in a separate container:
docker run -it --rm --link some-postgres:postgres postgres psql -h postgres -U postgres
# can use the host "postgres" since container links will populate /etc/hosts

What are the docker run lines that you are doing?

Another variant to the recommended version of @yosifkit
sudo docker run --name pt -t -p 0.0.0.0:5555:5432 -d postgres

to expose the container internal port 5432 to port 5555 on your host machine.

renzok, your advice is helped. i did change port to 5555 and pgadmin start connect to database. thank you!

good to hear :) I guess then that this issue is then good to be closed.

authors: Starefossen brizjin renzok yosifkit

repo: docker-library/postgres