text:
Remove migration output from setup, including test

fix #1023
I'm not 100% sure why but it's showing I added #[test] and [cfg(feature="postgres")] to the spec below it instead of to itself but I didn't 🤔 ...
Well I ran the spec and got it to pass. Let me know if something isn't up to par and I'll get it resolved.

I'm not 100% sure why but it's showing I added #[test] and [cfg(feature="postgres")] to the spec below it instead of to itself but I didn't 🤔 ...

It's just how git does things, sometimes it's a bit messed up, but it looks ok to me

Done @Eijebong 💪

Ty guys. That was my first non-documentation PR to an open source project. Thanks for being so helpful. @Eijebong @sgrif

authors: Eijebong juliusdelta

repo: diesel-rs/diesel


text:
Application crash due to the enhanced debug mode

Hi :)
With pull request #53 you've introduced an enhanced debug mode which is enabled in our pipelines by default to see a more verbose log.
Since the update a few hours ago, we've issues with the changes you made for the enhanced debug mode:
{"action":"generate report","data":{"provide":"gitlab"}}
{"action":"sonar get task status","data":"/api/ce/activity"}
{"action":"[axios] request","data":{"transitional":{"silentJSONParsing":true,"forcedJSONParsing":true,"clarifyTimeoutError":false},"transformRequest":[null],"transformResponse":[null],"timeout":0,"xsrfCookieName":"XSRF-TOKEN","xsrfHeaderName":"X-XSRF-TOKEN","maxContentLength":-1,"maxBodyLength":-1,"headers":{"common":{"Accept":"application/json, text/plain, */*"},"delete":{},"get":{},"head":{},"post":{"Content-Type":"application/x-www-form-urlencoded"},"put":{"Content-Type":"application/x-www-form-urlencoded"},"patch":{"Content-Type":"application/x-www-form-urlencoded"},"Authorization":"Bearer [MASKED]"},"method":"get","url":"[REDACTED]/api/ce/activity?component=[REDACTED]&onlyCurrents=true"}}
TypeError: Converting circular structure to JSON
    --> starting at object with constructor 'ClientRequest'
    |     property 'socket' -> object with constructor 'TLSSocket'
    --- property '_httpMessage' closes the circle
    at JSON.stringify (<anonymous>)
    at Logger.debug (/usr/local/lib/node_modules/sonar-quality-gate/build/utils/log.js:18:26)
    at onResponse (/usr/local/lib/node_modules/sonar-quality-gate/build/http/interceptors.js:14:17)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)

After the error the whole application crashes and doesn't continue decorating the merge requests in GitLab.

@jsproede
i have pushed code for fix this bug #56 , please check it again.

Hi @dieuhd,
sorry for the delay. The issue has been fixed with the changes you've pushed. Thank you! :)

source: gharchive/issue

authors: dieuhd jsproede

repo: dieuhd/sonar-quality-gate

license: MIT

license_type: permissive

license_source: github-api


text:
Add selection controls size tokens to Figma set

With how browsers round css pixels and fluid sizing we get skewed circles.
We have thus decided to add a separate token set using only rem values for sizing as they will not fluctuate as much due to viewport sizes.
Suggest naming these sizing tokens something with selection controls, ref: https://m1.material.io/components/selection-controls.html#
Suggest token sizes using

size
rem
base@16px

small
1.5rem
24px

medium
1.75rem
28px

large
2rem
32px

Decisions from #1444 might change this

This will be solved as part of #1444

authors: mimarz mrosvik

repo: digdir/designsystemet


text:
Etablere Azure Servicebus som kømekanisme

Azure Service Bus skal benyttes som meldingsbus, primært for CDC fra Postgres men også potensielt andre ting på sikt. Vi bruker MassTransit.io som abstraksjonslag.
Vi må se på en løsning for å sikre god DX her. Kanskje en eller annen memory-based løsning i MT.
- [ ] Fjerne alle spor av RabbitMQ
- [ ] Sette opp Bicep for oppsett av infra i Azure

Se også

https://digdir.github.io/dialogporten/#systemarkitektur
#49
#51

Starter med in-memory så får vi se hvor godt det funker da..! Kan jo eventuelt se på den hacken her og om vi blir misfornøyd https://devopsifyme.com/simple-azure-service-bus-emulator-finally-here/
Eller så kan vi joine alle andre som har venta siden 2018 på en emulator https://github.com/Azure/azure-service-bus/issues/223

authors: arealmaas elsand

repo: digdir/dialogporten


text:
"Splash Screen" changes

[ ] Add a prominent Open Manifest call to action. So the default view is Open Manifest, New Manifest, Recent Manifests and each of them are the same kind of size. The first one obviously only needs one box, with something "open-y" - a curving arrow?
[ ] Do we need the "Oops" view? Can't that just be the default view above?
[ ] Remove the copy, and put it in a modal that is opened from Help -> About.

Simplify to list of options - a larger version of the File menu.
Open
New
Recent
@diegolago to mock up.

I think some visual tidy should be improved here

[ ] Max width on the getting started section
[ ] Switch to list for recently opened resources

authors: stephenwf tomcrane

repo: digirati-co-uk/iiif-manifest-editor


text:
only one proposition per opinion

Is your feature request related to a problem? Please describe.
any user could spam others by sending them lots of propositions.
Describe the solution you'd like
Once the first proposition was sent. The button "send proposition" should disappear.
Additionally, it should not be possible to send more than one proposition.
Describe alternatives you've considered
The user could be promted with a error message. But why show him a button if he is not allowed to click it?
Additional context
If there is not many opinion, it is hard to navigate to the "proposition" button.
If there is a lot of opinions, it is even harder to find a good view on a specific proposition button.

done.

authors: Iwan-Ittermann dptalks

repo: digital-peace-talks/DPT-server


text:
Pusher tries to reconnect after coming out of sleep on PC

I am writing a WPF .NET application and have been using Pusher for the websocket implementation. I would call this issue an edge case but it's happening often enough that I really want to fix it. Whenever the PC goes to sleep and then returns out of sleep, the app crashes and it looks to be due to Pusher trying to reconnect the socket even though it thinks it is already connected. Here is the stack trace from the Event Viewer:
--FROM CLIENT-- INFO: System.Exception: The socket is connected, you neednt' connect again!
at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
at Pusher.Pusher.<<.ctor>b__2>d__5.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
at System.Runtime.CompilerServices.AsyncMethodBuilderCore.<>c.b__6_1(Object state)
at System.Threading.QueueUserWorkItemCallback.WaitCallback_Context(Object state)
at System.Threading.ExecutionContext.RunInternal(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, ContextCallback callback, Object state, Boolean preserveSyncCtx)
at System.Threading.QueueUserWorkItemCallback.System.Threading.IThreadPoolWorkItem.ExecuteWorkItem()
at System.Threading.ThreadPoolWorkQueue.Dispatch()
at System.Threading._ThreadPoolWaitCallback.PerformWaitCallback() AT 6/29/2017 9:12:36 PM

The socket is connected, you neednt' connect again! I found in the SuperSocket source code which Pusher.NET uses. Any guidance here would be immensely helpful. I have everything wrapped in exception handling but the app still crashes and I'm not sure why. It also doesn't help that I cannot personally reproduce this issue.

I ended up having to create a method that pings the internet and if the app can't reach google.com, then I disconnect from Pusher. I would have assumed Pusher.NET had some way to handle this, but it doesn't look like it does.

@johnmurphy01 Thanks for letting me know of this issue. As you may be able to tell from this repo, this is no longer under active development. We made this library for a client's project, a project that has since been discontinued. We are therefore not maintaining it, but it should work.
If you do have the possibility to implement a proper fix for your issue, please let me know and I will attempt to create a release for it.

@vegardlarsen thanks for replying. As I mentioned, I had to implement a method that determines whether my app is connected to the internet. If it is not, I disconnect from pusher. Once the network is re-established, I re-connect to the websocket using Pusher.
Ideally, I'd love for Pusher.NET to handle this for me. But I can always fork the project if it bothers me too much. Thanks again!

Re-opened and created a PR(#5) to fix

I have merged your pull request and published version 0.3.2 with the changes to Nuget.

Thanks!

license: mit

license_source: bigquery


text:
Broken Link Checker shows links in languages that are disabled

Describe the Bug
The broken link checker shows broken links that still exist in disabled languages
Steps to Reproduce

Go to https://admin.integreat-app.de/deutschlandvorlage/analytics/linkcheck/invalid/
Hover over the link for the source or click on it
See that it links to bulgarian content

Expected Behavior
Link should not show up bcs language is disabled (not just invisible)
Actual Behavior
Link in Bulgarian is shown
Additional Information

I'm pretty sure that this should be an infra ticket.
We can still check for bugs in the code base, but I think the race conditions are responsible for this inconsistent state and we need to rebuild the link database on the production server.

Ahh, but if it is only limited to disabled languages, it might be actually a valid ticket, sorry.
I guess we did not think about deleting all links when disabling a language and to re-create them when enabling a language? 🤔

authors: osmers timobrembeck

repo: digitalfabrik/integreat-cms


text:
🛑 Corbin.World is down

In 4395f9b, Corbin.World (https://corbin.world) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Corbin.World is back up in 85ef7b3.


text:
Skip integration tests when required executables are missing

If the running host is missing some of the required executables,
then skip running the integration tests with a reason as to why.
This changes the experience when either docker or git is not present to be something like this (ran via pytest -rsxvv):
tests/test_api.py .....                                                                                [  1%]
tests/test_app.py ....                                                                                [  3%]
tests/test_circuits.py ..............                                                                             [  7%]
tests/test_dcim.py .............................................................................................. [ 39%]
.............................................................................                                     [ 65%]
tests/test_ipam.py ................................                                                               [ 76%]
tests/test_tenancy.py ......                                                                                [ 78%]
tests/test_users.py ..........                                                                                [ 82%]
tests/test_virtualization.py ...............                                                                      [ 87%]
tests/integration/test_dcim.py s                                                                                [ 87%]
tests/integration/test_pytest_docker.py s                                                                         [ 88%]
tests/integration/test_dcim.py s                                                                                [ 88%]
tests/integration/test_pytest_docker.py s                                                                         [ 88%]
tests/integration/test_dcim.py s                                                                                [ 89%]
tests/integration/test_pytest_docker.py s                                                                         [ 89%]
tests/integration/test_dcim.py s                                                                                [ 89%]
tests/integration/test_pytest_docker.py s                                                                         [ 90%]
tests/unit/test_endpoint.py ....                                                                                [ 91%]
tests/unit/test_query.py ..                                                                                [ 92%]
tests/unit/test_request.py .                                                                                [ 92%]
tests/unit/test_response.py ......................                                                                [100%]

================================================ short test summary info ================================================
SKIPPED [8] tests/integration/conftest.py:261: docker executable was not found on the host
============================================ 286 passed, 8 skipped in 4.46s =============================================

Instead of the previous very verbose and unclear errors.

@markkuleinio Does this make sense?

Makes sense. :+1:

Sorry for the late reply. I don't see any skipped tests, just lots of failures ("EEEEEEEEEEEEE") when running pytest locally in the root pynetbox directory. Can you be more specific how this skipping should work?
...
ERROR tests/integration/test_ipam.py::TestVRF::test_update_fixture[netbox v2.10] - FileNotFoundError: [Errno 2] No such file or directory: 'docker': 'docker'
ERROR tests/integration/test_ipam.py::TestVRF::test_get_fixture_by_id[netbox v2.10] - FileNotFoundError: [Errno 2] No such file or directory: 'docker': 'docker'
ERROR tests/integration/test_ipam.py::TestVRF::test_get_fixture_by_kwarg[netbox v2.10] - FileNotFoundError: [Errno 2] No such file or directory: 'docker': 'docker'
ERROR tests/integration/test_ipam.py::TestVRF::test_filter_fixture[netbox v2.10] - FileNotFoundError: [Errno 2] No such file or directory: 'docker': 'docker'
===================================================================== 83 passed, 392 errors in 28.22s ======================================================================

In your own tests, did it work?

I didn't test it, but looks like we need to move it into the first function to get it to work. I'll PR it real fast.

source: gharchive/pull-request

authors: markkuleinio raddessi zachmoody

repo: digitalocean/pynetbox


text:
Atomic operation for single coil requests

I am looking for a modbus library to integrate within an JavaEE application server.
Basically I need a stateless option to for a single coil request:
        ModbusTcpMasterConfig config = new ModbusTcpMasterConfig.Builder(endpoint)
            .setPort(PORT)
            .build();
        ModbusTcpMaster master = new ModbusTcpMaster(config);
        WriteSingleCoilRequest req = new WriteSingleCoilRequest(channel, enabled);
        master.sendRequest(req, 0).thenAccept(response ->
        {
            ReferenceCountUtil.release(response);
        });

This method can be called multiple times and even in parallel. I noticed that after this the library still have some open threads. Calling Modbus.releaseSharedResources(); does not work for parallel or fast consecutive calls.
I think it would be good if in a method like disconnect all resources are released and provide a thread safe operation.

I've got no problem making releaseSharedResources() more thread safe, but it's meant to be called on application or JVM shutdown, not for every request you service.
These resources are relatively heavy and meant to be shared by many instances of ModbusTcpMaster.
Further, if your service truly needs to be stateless, then by definition you can't be using these shared resources. Instead, you need to create your own instances of each of these per request and set them on the config builder. This will prevent it from reaching for and possibly instantiating the shared instances.
You'll suffer a performance hit for this. Things like the event loop, and especially the wheel timer, are meant to be shared across an application, not created and stopped constantly.

Thank you for the fast response @kevinherron . I understand your concept better now. My use case is only to write some single coils and I was looking for a lightweight way to do this like open TCP connection, send command, close connection, release resources in a thread safe way.

Try creating your own instances of the shared resources and shutting them down. It may still be performant enough for your usage.

Closing this. I fixed up the synchronization and improved the shared resource stuff a bit for 1.2.0, but you'll still need to configure and manage your own for truly stateless operation.

authors: kevinherron simdevmon

repo: digitalpetri/modbus


text:
🛑 android-8.1.0_r81 is down

In 4ce2904, android-8.1.0_r81 (http://aospxref.com/android-8.1.0_r81/) was down:

HTTP code: 404
Response time: 247 ms

Resolved: android-8.1.0_r81 is back up in 860e07c after 9 minutes.

repo: dimenspace/aosp-uptime


text:
🛑 android-11.0.0_r21 is down

In 173f918, android-11.0.0_r21 (http://aospxref.com/android-11.0.0_r21/) was down:

HTTP code: 404
Response time: 202 ms

Resolved: android-11.0.0_r21 is back up in 50ff31e after 51 minutes.


text:
Uncaught Error: Syntax error, unrecognized expression:

There is a problem with parsing html from ajax content,
Console return
Uncaught Error: Syntax error, unrecognized expression: <my html ...>

to avoid this situation you should replace
data:data,

with:
data: $.parseHTML(data),

in line 1033

I got the same, $.parseHTML() solved it.
It was already a HTML string:
      items: {
        src: $.parseHTML(html),
        type: 'inline'
      },

Why is $.parseHTML() still required?

Or if you get this error on inline content it could be because you've simply put # in the href attribute.

authors: mariusa r4fx texelate

repo: dimsemenov/Magnific-Popup


text:
hdf5 locking fix

For some reason a fresh install of dingo and executing dingo_pipe on a .ini file gives an error stating that a BilbyMainInput doesn't have a disable_hdf5_locking attribute. I'm not actually sure if this should be set from the command line arguments via args.disable_hdf5_locking, but setting it to None fixes the issue.

Could this be a problem that is specific to AEI clusters? Do you get the same issue on your laptop or on CIT?

I have have implemented changes corresponding to
https://git.ligo.org/lscsoft/bilby_pipe/-/merge_requests/529
https://git.ligo.org/lscsoft/bilby_pipe/-/merge_requests/534
dingo_pipe requires bilby_pipe >= 1.0.9 to work now, so this is a requirement in pyproject.toml.
Please check that dingo_pipe works, since I have not tested these changes.

Please check that dingo_pipe works, since I have not tested these changes.

Works on my local machine

authors: mpuerrer nihargupte-ph stephengreen

repo: dingo-gw/dingo


text:
Rewrite gem to use Arel instead of concatenating strings

This pull request rewrites acts_as_geolocated methods to use Arel instead of concatenating strings together. This should prevent issues like #13 in the future and is less error-prone.
All tests have been checked and they are passing correctly.

Thanks again, sorry for the delay to merge


text:
All FilteringContainer props' types are required

Hi @diogofcunha! I started using your library, but came across the following issue: FilteringContainerProps interface in index.d.ts file has all props set as required, despite some of them are marked as optional here https://diogofcunha.github.io/react-virtualized-tree/#/docs/filtering. Is it a bug?

Hello Salazar.
They are wrong, can you please open a pr to fix it? Otherwise I'll do it in the next few days

Hi @Salasar.
Thanks for reporting this issue.
This is fixed and was released as part of 2.3.2

@diogofcunha sorry, I didn't have time to fix it on my own. Thank you for the fix!

authors: Salasar diogofcunha

repo: diogofcunha/react-virtualized-tree


text:
Added ClearButton

Contributes to #44

Current coverage is 45.65% (diff: 0.00%)

Merging #48 into master will decrease coverage by 1.10%

@@             master        #48   diff @@
==========================================
  Files            25         26     +1
  Lines           866        887    +21
  Methods           0          0
  Messages          0          0
  Branches          0          0
==========================================
  Hits            405        405
- Misses          461        482    +21
  Partials          0          0

Powered by Codecov. Last update 93a13ab...90307fa

authors: codecov-io diogot

repo: diogot/MyWeight


text:
🛑 URL Shortener is down

In 0970149, URL Shortener ($SHORTENER) was down:

HTTP code: 0
Response time: 0 ms

Resolved: URL Shortener is back up in 33b13ad after 12 hours, 35 minutes.

repo: dipoapp/uptime


text:
crust_node example output and usage comments are unhelpful.

The comments in crust_node.rs state e.g.

// starting second node : cargo run --example crust_node -- 5883 -b 0.0.0.0:5983

The output from running a crust_node in server mode by doing
target\debug\examples\crust_node.exe -o 6666

yields the following output:

Connection manager now listening on TCP socket 0.0.0.0:6666

However, when connecting another node, we need to use the server's actual IP address, not 0.0.0.0.

Using 0.0.0.0 doesn't seem to be an issue on OS X, and probably isn't a problem on Linux either.

This means all IP addresses, I think with beacon on a LAN we will be OK, but the node should find it's own IP address for those who connect to it as well. We perhaps require the sniffing logic from rudp and the IANA addresses for checking external IP.

using 0.0.0.0 or 127.0.0.1 or the actually ip address depends on what has been printed out in "add new node ", which might be OS dependent.

authors: Fraser999 dirvine maqi

repo: dirvine/crust


text:
[Bug] Cannot apply permissions for subcommands with more than one role per guild

I have several subcommands in cogs like this:
base_permissions = {
    guild_a.id: [
        {'id': role_a.id, 'type': SlashCommandPermissionType.ROLE, 'permission': True},
        {'id': role_b.id, 'type': SlashCommandPermissionType.ROLE, 'permission': True},
        {'id': role_c.id, 'type': SlashCommandPermissionType.ROLE, 'permission': True},
    ]
}
...
@cog_subcommand(
    name='subcommand'
    base='base_command',
    base_permissions=base_permissions,
    base_default_permission=False,
    guild_ids=list(base_permissions.keys()),
)
async def subcomand(ctx):
    await ctx.reply('whatever')

When I start up the bot, I get an HTTP Error from discord
future: <Task finished name='Task-1' coro=<SlashCommand.sync_all_commands() done, defined at C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\discord-party-dingomata-gAKNqU_v-py3.9\lib\site-packages\discord_slash\client.py:364> exception=HTTPException('400 Bad Request (error code: 50035): Invalid Form Body\nIn 0.permissions: Must be 10 or fewer in length.\nIn 1.per
missions: Must be 10 or fewer in length.')>
Traceback (most recent call last):
  File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\discord-party-dingomata-gAKNqU_v-py3.9\lib\site-packages\discord_slash\client.py", line 490, in sync_all_commands
    await self.req.update_guild_commands_permissions(scope, new_perms)
  File "C:\Users\user\AppData\Local\pypoetry\Cache\virtualenvs\discord-party-dingomata-gAKNqU_v-py3.9\lib\site-packages\discord\http.py", line 254, in request
    raise HTTPException(r, data)
discord.errors.HTTPException: 400 Bad Request (error code: 50035): Invalid Form Body
In 0.permissions: Must be 10 or fewer in length.
In 1.permissions: Must be 10 or fewer in length.

This only happens when I define more role for one server - I can have multiple guilds, but not more than one role each. All slash commands are successfully registered, but for servers where more than one role is given permission, the permission is not applied.

I have found out that this is actually a limitation of the Discord API that this package is not respecting. Will open new issue.


text:
chore: tags

add message-formatting (timestamp) tag

Didn't see it already existed! My bad.

repo: discordjs/discord-utils-bot


text:
rules

What is your username and discriminator?
=Lewd King= Curly Bryce#6326
Why did you join Discord Linux?
For support and because I use Linux
Where did you hear about Discord Linux?
From a discord issue ticket
What is the passphrase that was assigned to you by the bot?
I have looked through this link multiple times and cannot find the passphrase
https://blargbot.xyz/output/511240873850191307

From simonizor#0420 :
That is not your passphrase. Please read the contents of that link to find your passphrase.

authors: CurlyBryce DiscordLinuxBot

repo: discordlinux/rules


text:
🛑 Distrochooser Frontend (beta) is down

In ae4d40a, Distrochooser Frontend (beta) (https://beta.distrochooser.de) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Distrochooser Frontend (beta) is back up in 77a2ae7 after 56 minutes.

repo: distrochooser/status


text:
OT 3.3.1 with Saxon-PE  9.9.1 fails in mapref

When I use the newest Saxon-PE release (9.9.1.2), DITA-OT 3.3.1 fails with java.lang.NoSuchMethodError. It woks fine with Saxon-PE 9.8.0.14.
[command-line]
bin/dita -f pdf -i docsrc/samples/sequence.ditamap -o out -l out/log.txt -v
[Log File:out/log.txt]
init:
dita2pdf2.init:
init-properties:
check-arg:
[mkdir] Created dir: C:\Users\toshi\AppData\Local\Temp\temp20190410101743125
log-arg:
[echo] *****************************************************************
[echo] * basedir = D:\DITA-OT\dita-ot-3.3.1
[echo] * dita.dir = D:\DITA-OT\dita-ot-3.3.1\bin..
[echo] * transtype = pdf
[echo] * tempdir = C:\Users\toshi\AppData\Local\Temp\temp20190410101743125
[echo] * outputdir = D:\DITA-OT\dita-ot-3.3.1\out
[echo] * clean.temp = true
[echo] * DITA-OT version = 3.3.1
[echo] * XML parser = Xerces
[echo] * XSLT processor = Saxon
[echo] * collator = ICU
[echo] *****************************************************************
[echo] #Ant properties
[echo] #Wed Apr 10 10:17:43 JST 2019
[echo] dita.plugin.org.dita.base.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.base
[echo] preprocess.chunk.skip=true
[echo] dita.temp.dir=C:\Users\toshi\AppData\Local\Temp\temp20190410101743125
[echo] dita.plugin.org.dita.troff.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.troff
[echo] args.grammar.cache=yes
[echo] dita.plugin.org.oasis-open.xdita.v0_2_2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.xdita.v0_2_2
[echo] dita.dir=D:\DITA-OT\dita-ot-3.3.1\bin\..
[echo] dita.plugin.org.dita.pdf2.axf.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.axf
[echo] dita.plugin.com.sophos.tocjs.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\com.sophos.tocjs
[echo] dita.plugin.org.oasis-open.dita.v1_3.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.dita.v1_3
[echo] dita.plugin.org.lwdita.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.lwdita
[echo] dita.plugin.org.dita.htmlhelp.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.htmlhelp
[echo] dita.output.dir=D:\DITA-OT\dita-ot-3.3.1\out
[echo] args.rellinks=nofamily
[echo] dita.plugin.org.dita.eclipsehelp.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.eclipsehelp
[echo] dita.plugin.org.dita.normalize.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.normalize
[echo] dita.plugin.org.dita.xhtml.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.xhtml
[echo] dita.plugin.org.dita.pdf2.xep.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.xep
[echo] dita.plugin.org.dita.specialization.dita11.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.specialization.dita11
[echo] dita.plugin.org.oasis-open.dita.v1_2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.dita.v1_2
[echo] dita.plugin.org.dita.pdf2.fop.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.fop
[echo] args.input=D:\DITA-OT\dita-ot-3.3.1\docsrc\samples\sequence.ditamap
[echo] args.xml.systemid.set=yes
[echo] dita.plugin.org.dita.pdf2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2
[echo] dita.plugin.org.dita.html5.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.html5
[echo] args.logdir=D:\DITA-OT\dita-ot-3.3.1\out
[echo] preprocess.copy-image.skip=true
[echo] *****************************************************************
build-init:
preprocess2.init:
[echo] *****************************************************************
[echo] * input = D:\DITA-OT\dita-ot-3.3.1\docsrc\samples\sequence.ditamap
[echo] *****************************************************************
ditaval-merge:
map-reader:
[map-reader] Using Xerces grammar pool for DTD and schema caching.
[map-reader] Processing file:/D:/DITA-OT/dita-ot-3.3.1/docsrc/samples/sequence.ditamap to file:/C:/Users/toshi/AppData/Local/Temp/temp20190410101743125/f7f208540ed460242874049c8cc5c8f4bcc5665b.ditamap
[map-reader] Serializing job specification
[job-helper] Processing C:\Users\toshi\AppData\Local\Temp\temp20190410101743125.job.xml to C:\Users\toshi\AppData\Local\Temp\temp20190410101743125\fullditamap.list
[job-helper] Loading stylesheet D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.base\xsl\job-helper.xsl
[delete] Deleting: C:\Users\toshi\AppData\Local\Temp\temp20190410101743125\fullditamap.list
map-mapref:
[mapref] Processing file:/C:/Users/toshi/AppData/Local/Temp/temp20190410101743125/f7f208540ed460242874049c8cc5c8f4bcc5665b.ditamap
Error: java.lang.NoSuchMethodError: net.sf.saxon.Controller.setMessageEmitter(Lnet/sf/saxon/event/Receiver;)V

The current release of DITA-OT requires 9.8 (which is bundled in the download) to function properly.
I encountered this same issue when testing with 9.9 a few months ago, and tracked it down to a method that moved from the Controller class into the XsltController class. Based on reading the latest Saxon docs, I think that when we update DITA-OT to ship with 9.9 we will have to switch from calling
net.sf.saxon.Controller.setMessageEmitter
to call
net.sf.saxon.XsltController.setMessageEmitter
I have not tried any of this and could be wrong, but a few months back I found the latest method documented here: http://www.saxonica.com/html/documentation/javadoc/net/sf/saxon/trans/XsltController.html#setMessageEmitter-net.sf.saxon.event.Receiver-
In the 9.8 docs it's part of the Controller class: https://www.saxonica.com/html/documentation9.8/javadoc/net/sf/saxon/Controller.html#setMessageEmitter-net.sf.saxon.event.Receiver-

More problem is here. My user must purchase Saxon-PE because they must call external Java function library from DITA-OT 3.2.1. When they use Saxon-PE 9.8.0.15, the build fails with following error (In my test environment: using OT 3.3.1):
[The Log FIle]
init:
dita2pdf2.init:
init-properties:
check-arg:
[mkdir] Created dir: C:\Users\toshi\AppData\Local\Temp\temp20190410115510437
log-arg:
[echo] *****************************************************************
[echo] * basedir = D:\DITA-OT\dita-ot-3.3.1
[echo] * dita.dir = D:\DITA-OT\dita-ot-3.3.1\bin..
[echo] * transtype = pdf
[echo] * tempdir = C:\Users\toshi\AppData\Local\Temp\temp20190410115510437
[echo] * outputdir = D:\DITA-OT\dita-ot-3.3.1\out
[echo] * clean.temp = true
[echo] * DITA-OT version = 3.3.1
[echo] * XML parser = Xerces
[echo] * XSLT processor = Saxon
[echo] * collator = ICU
[echo] *****************************************************************
[echo] #Ant properties
[echo] #Wed Apr 10 11:55:10 JST 2019
[echo] dita.plugin.org.dita.base.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.base
[echo] preprocess.chunk.skip=true
[echo] dita.temp.dir=C:\Users\toshi\AppData\Local\Temp\temp20190410115510437
[echo] dita.plugin.org.dita.troff.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.troff
[echo] args.grammar.cache=yes
[echo] dita.plugin.org.oasis-open.xdita.v0_2_2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.xdita.v0_2_2
[echo] dita.dir=D:\DITA-OT\dita-ot-3.3.1\bin\..
[echo] dita.plugin.org.dita.pdf2.axf.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.axf
[echo] dita.plugin.com.sophos.tocjs.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\com.sophos.tocjs
[echo] dita.plugin.org.oasis-open.dita.v1_3.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.dita.v1_3
[echo] dita.plugin.org.lwdita.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.lwdita
[echo] dita.plugin.org.dita.htmlhelp.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.htmlhelp
[echo] dita.output.dir=D:\DITA-OT\dita-ot-3.3.1\out
[echo] args.rellinks=nofamily
[echo] dita.plugin.org.dita.eclipsehelp.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.eclipsehelp
[echo] dita.plugin.org.dita.normalize.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.normalize
[echo] dita.plugin.org.dita.xhtml.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.xhtml
[echo] dita.plugin.org.dita.pdf2.xep.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.xep
[echo] dita.plugin.org.dita.specialization.dita11.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.specialization.dita11
[echo] dita.plugin.org.oasis-open.dita.v1_2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.oasis-open.dita.v1_2
[echo] dita.plugin.org.dita.pdf2.fop.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2.fop
[echo] args.input=D:\DITA-OT\dita-ot-3.3.1\docsrc\samples\sequence.ditamap
[echo] args.xml.systemid.set=yes
[echo] dita.plugin.org.dita.pdf2.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.pdf2
[echo] dita.plugin.org.dita.html5.dir=D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.html5
[echo] args.logdir=D:\DITA-OT\dita-ot-3.3.1\out
[echo] preprocess.copy-image.skip=true
[echo] *****************************************************************
build-init:
preprocess2.init:
[echo] *****************************************************************
[echo] * input = D:\DITA-OT\dita-ot-3.3.1\docsrc\samples\sequence.ditamap
[echo] *****************************************************************
ditaval-merge:
map-reader:
[map-reader] Using Xerces grammar pool for DTD and schema caching.
[map-reader] Processing file:/D:/DITA-OT/dita-ot-3.3.1/docsrc/samples/sequence.ditamap to file:/C:/Users/toshi/AppData/Local/Temp/temp20190410115510437/f7f208540ed460242874049c8cc5c8f4bcc5665b.ditamap
[map-reader] Serializing job specification
[job-helper] Processing C:\Users\toshi\AppData\Local\Temp\temp20190410115510437.job.xml to C:\Users\toshi\AppData\Local\Temp\temp20190410115510437\fullditamap.list
[job-helper] Loading stylesheet D:\DITA-OT\dita-ot-3.3.1\plugins\org.dita.base\xsl\job-helper.xsl
[delete] Deleting: C:\Users\toshi\AppData\Local\Temp\temp20190410115510437\fullditamap.list
map-mapref:
[mapref] Error on line 8 column 80 of mapref.xsl:
[mapref]   Malformed URL plugin:org.dita.base:xsl/preprocess/maprefImpl.xsl(base
[mapref]   file:/D:/DITA-OT/dita-ot-3.3.1/plugins/org.dita.base/xsl/preprocess/mapref.xsl): unknown
[mapref]   protocol: plugin
Error: java.lang.RuntimeException: javax.xml.transform.TransformerConfigurationException: net.sf.saxon.s9api.SaxonApiException: Malformed URL plugin:org.dita.base:xsl/preprocess/maprefImpl.xsl(base file:/D:/DITA-OT/dita-ot-3.3.1/plugins/org.dita.base/xsl/preprocess/mapref.xsl)
The cause was very curious. catalog-dita.xml was generated with xmlns='' entries for plugin.
[Catalog  File: dita-ot-3.3.1/plugins/org.dita.base/catalog-dita.xml]
<?xml version="1.0" encoding="UTF-8"?><!--
This file is part of the DITA Open Toolkit project.

Copyright 2004, 2005 IBM Corporation

See the accompanying LICENSE file for applicable license.
--><catalog xmlns="urn:oasis:names:tc:entity:xmlns:xml:catalog" xmlns:dita="http://dita-ot.sourceforge.net">

  <rewriteURI uriStartString="platform:" rewritePrefix="../../"/>

  <rewriteURI xmlns="" uriStartString="plugin:org.dita.htmlhelp:" rewritePrefix="../org.dita.htmlhelp/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.base:" rewritePrefix="../org.dita.base/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.pdf2.xep:" rewritePrefix="../org.dita.pdf2.xep/"/><rewriteURI xmlns="" uriStartString="plugin:com.sophos.tocjs:" rewritePrefix="../com.sophos.tocjs/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.pdf2.fop:" rewritePrefix="../org.dita.pdf2.fop/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.eclipsehelp:" rewritePrefix="../org.dita.eclipsehelp/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.specialization.dita11:" rewritePrefix="../org.dita.specialization.dita11/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.normalize:" rewritePrefix="../org.dita.normalize/"/><rewriteURI xmlns="" uriStartString="plugin:org.oasis-open.xdita.v0_2_2:" rewritePrefix="../org.oasis-open.xdita.v0_2_2/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.troff:" rewritePrefix="../org.dita.troff/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.pdf2.axf:" rewritePrefix="../org.dita.pdf2.axf/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.html5:" rewritePrefix="../org.dita.html5/"/><rewriteURI xmlns="" uriStartString="plugin:org.lwdita:" rewritePrefix="../org.lwdita/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.xhtml:" rewritePrefix="../org.dita.xhtml/"/><rewriteURI xmlns="" uriStartString="plugin:org.dita.pdf2:" rewritePrefix="../org.dita.pdf2/"/><rewriteURI xmlns="" uriStartString="plugin:org.oasis-open.dita.v1_2:" rewritePrefix="../org.oasis-open.dita.v1_2/"/><rewriteURI xmlns="" uriStartString="plugin:org.oasis-open.dita.v1_3:" rewritePrefix="../org.oasis-open.dita.v1_3/"/>

   <nextCatalog catalog="../org.oasis-open.dita.v1_3/catalog.xml"/><nextCatalog catalog="../org.oasis-open.dita.v1_2/catalog.xml"/><nextCatalog catalog="../org.dita.pdf2/cfg/catalog.xml"/><nextCatalog catalog="../org.dita.specialization.dita11/catalog.xml"/><nextCatalog catalog="../org.oasis-open.xdita.v0_2_2/catalog-dita.xml"/>

</catalog>

As a result my user is very confusing.

In addition to the API changes Robert mentioned, 9.9 changes the API for registering extension functions, which will require a change to the current code for automatic extension function registration for plugins and the corresponding code in plugins that contribute extension functions.

The namespace issue in catalogs has been fixed in #3287. Can this be closed, given that 3.4 will use Saxon 9.9?

Can this be closed, given that 3.4 will use Saxon 9.9?

Yes, I agree!

authors: ToshihikoMakita drmacro jelovirt robander

repo: dita-ot/dita-ot


text:
Aliases don't get copied correctly when using multi-language

I have a site that has pages in 5 different languages. In the home page I have 2 style plugins which I add to 3 other pages using alias (with its corresponding children). But when I copied those 3 pages to other languages using the Copy all plugins from English, the alias that gets generated is pointing to the plugins in the English homepage, not the corresponding new language's homepage.
To fix this, I have to delete all the aliases I have inserted on the different pages and recreate the alias in the language's homepage and the re-paste them where I want them so they don't point to the wrong plugin.

Hello @lmanzurv ,
This was a design decision.
The alias  plugin allows content managers to reference plugins regardless of their language.
Also when the alias plugin is copied it has way to know which plugin represents it's aliased plugin in the target language as plugins are not related.
I recommend using placeholder aliases instead of plugin aliases, the advantage is that the same placeholder is used for all translations in a page and so the alias of a placeholder will only render plugins in the current language.

authors: czpython lmanzurv

repo: divio/django-cms


text:
Move urls.SLUG_REGEXP to constants

This fixes the problem with Django 1.9 system checks. In a few words:
importing from cms.urls triggers app resolver mechanism that fails
if there were no database tables for contrib.sites created yet.
It catches DoesNotExist but not OperationalError. Moving this
constant out to a different location seems to be a better fix.

:+1:

LGTM. A lot of circular import fixes since that comments.
Merged through #4995

Fixed by #4995

Coverage decreased (-0.0009%) to 89.115% when pulling 10a6b3432d7fe3ccd34cb5dbbf62e30b1efd7abe on mikek:issue/django-1-9-checks-workaround into 52f61ac4cbfe20b55de207cc866356e39e4a676a on divio:develop.

Coverage decreased (-0.0009%) to 89.115% when pulling 10a6b3432d7fe3ccd34cb5dbbf62e30b1efd7abe on mikek:issue/django-1-9-checks-workaround into 52f61ac4cbfe20b55de207cc866356e39e4a676a on divio:develop.

authors: Chive coveralls mikek yakky


text:
Feature/drag and drop widget

This PR implements drag and drop for the widgets in the structure mode.
Please do not review dropzone.scss.
JS is also outdated, just tried to leave the old styling (because it can be used by some other plugins).

:+1:

authors: Flight ruslanmalogulko

repo: divio/django-filer


text:
allow JS tokens and bundle to support relative imports

Example
/#project=rVRRb4IwEP4rpHtYYkhxL0u2PW1L3Oveh1kqFC1CS67FaAj/fddWBNSpS3ZRaK/ffe19vaMhiZKZWNJcK0meSRPLIIiJVjUkPCbPwZd1WJdRay51lKhCQTSZRBPa5Dq0YW1MLGge+tiqYCZTUGob3nThiR7OHSEwqS3wA1Rd2cU9KuwxFfBMbP2aTkdLi1oU6SczK7/qphHGRyNUJgquB3lYa/qhhaRcGyGZEUp6qu8NA8EWGEjH59lT4pGZOZw3OqC9DN7abjj3gzbshMhv0SHX13PN/yHVPtP8cqI52zCdgKhMxPXjxUztCx8tCcc1s2Ca4zZYZKKsFJggUyrIQJXBPaURThABCLh/iSGW9s+3DpfyjNWFwWTAV5jj8yqC35RSigRhN0WJcLMRwjqXwHZHTusuxHJlTvyBk6eouRfg7t0ZZj7AtOGYqeSpqMurVE/OLlOlDNZXiR6cHRH1kyFnTICnZ5If0s1mU7Qh3ZhhCZzLKxzTqWUZcXTD/cC9TqojU9IcfYQG1yy7ipTmqHv6iz5/oMaxUFd8qAB1S21fwO2ggzTHjyEKv7uV0AlykdJwMOJPjGxHXUGe0v7aX33voHjnmyaw3YZHcKR209e3d/zFZH8b7Q8=
Put that after the deploy preview link and you should see a working example where you can use JS tokens in conjunction with JSON tokens, and you can even do relative imports to other files.
Beware that if you import X into Y, that only Y is matched by the tokens glob in the SD config, otherwise you will get duplicate tokens from both X and Y being matched.

Aah I like that idea actually, will make a PR for that 👍  I agree it's a bit out of place there location-wise, and also the fact that it stays

repo: divriots/style-dictionary-playground


text:
🛑 ArtChart is down

In dc6cab6, ArtChart (https://artchart.net/en) was down:

HTTP code: 504
Response time: 15948 ms

Resolved: ArtChart is back up in a79befe after 4 minutes.

repo: divtiply/artupptime


text:
🛑 ArtChart is down

In 9c1f0e3, ArtChart (https://artchart.net/en) was down:

HTTP code: 504
Response time: 15664 ms

Resolved: ArtChart is back up in 7ca206d after 5 minutes.


text:
freedomsponsors is not a big bounty platform

freedomsponsors dot org is a gambling website

also:
safehats dot com
bugbountyhq dot com
hackerhive dot io
antihack dot me
aren't reachable

repo: djadmin/awesome-bug-bounty


text:
Add pencil buttons to ticket view for easier assignment and priority …

…setting

Right now the buttons just jump down to the bottom section, which isn't very modern (modern web design would make the field transform into an editable one) but I actually find it to be good, because then you're down right next to the submit button when preforming a quick change.

Fixes  https://github.com/django-helpdesk/django-helpdesk/issues/677 I guess.

Codecov Report

Merging #705 into develop will not change coverage.
The diff coverage is n/a.

@@           Coverage Diff            @@
##           develop     #705   +/-   ##
========================================
  Coverage    69.39%   69.39%
========================================
  Files           42       42
  Lines         4061     4061
  Branches       497      497
========================================
  Hits          2818     2818
  Misses        1100     1100
  Partials       143      143

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7699c48...c33d6d2. Read the comment docs.

authors: codecov-io timthelion

repo: django-helpdesk/django-helpdesk


text:
SuccessResponseView overrides the chosen shipping method

I have been tracking down why my orders come through with the wrong shipping method code, but correct method name.
It's because in express checkout we have this code:
    def get_shipping_method(self, basket, shipping_address=None, **kwargs):
        """
        Return the shipping method used
        """
        if not basket.is_shipping_required():
            return NoShippingRequired()

        # Instantiate a new FixedPrice shipping method instance
        charge_incl_tax = D(self.txn.value('PAYMENTREQUEST_0_SHIPPINGAMT'))

        # Assume no tax for now
        charge_excl_tax = charge_incl_tax
        method = FixedPrice(charge_excl_tax, charge_incl_tax)
        name = self.txn.value('SHIPPINGOPTIONNAME')

        if not name:
            session_method = super(SuccessResponseView, self).get_shipping_method(
                basket, shipping_address, **kwargs)
            if session_method:
                method.name = session_method.name
        else:
            method.name = name
        return method

Is there a reason we don't attempt to use the real shipping method class and instead instantiate a FixedPrice and set attributes on it?  Is this deliberately to avoid recalculating the price?  We have the basket object here and in build_submission method, can we not trust the real shipping method object to calculate the same price again?
If we're going to do it like above it should also set the code attribute at least

Closed by #107. Thank you for the great analysis and PR.

authors: anentropic maikhoepfel

repo: django-oscar/django-oscar-paypal


text:
Remove unused OSCAR_SETTINGS setting

OSCAR_SETTINGS just duplicates all of Oscar's settings into a dictionary, which is then not used anywhere. Looking back at the project history, it seems like it hasn't been used for a few years.

Anyway needs mention in the release docs.

I've added a mention to the release notes.

repo: django-oscar/django-oscar


text:
Using custom request

Hi, Is there a way to access the request session that makes the call to salesforce in a view?
I am trying to use this package with OpenId credentials.  Ultimately would like to do something like this:
def my_accounts(request):
    session = requests.Session()
    session.headers.update(...)
    Account.objects(session=session).all()

Thanks

I’m not fully up to date on what OpenID integration SFDC provides, but I’m not sure this would be a supported use case.
You’d still want to associate the connection with an “Application” in Salesforce, so you’d still end up getting OAuth keys generated.
Also, I personally prefer relying on the security code instead of whitelisting IPs, and I’m not sure how this would work if you are skipping the Salesforce authenticator.
@hynekcer may have deeper insight...

Thanks for the quick response.
I am using this authentication flow here
But I'm not suggesting managing the authentication flow in this package.  I'm just looking for a way to update the request session from a view post authentication.
Currently I'm using:
from simple_salesforce import Salesforce
def my_accounts(request):
    sf = Salesforce(instance=request.user.instance, session_id=request.user.session_id)
    sf.query("SELECT ...")
    ...

But using simple-salesforce means writing lots of SOQL.
As a backup plan I may go this route
from simple_salesforce import Salesforce
def my_accounts(request):
    sf = Salesforce(instance=request.user.instance, session_id=request.user.session_id)
    accounts = Account.objects.all()
    sf.query(accounts.query)
    ...

Interesting...I have an idea but I'm not near my dev workstation, so you might have to experiment a bit.
The requests session object in question is deep inside the RawConnection object that simulates the Python DB API. I think you can do something like this:
from django.db import connections

conn = connections['salesforce']
conn.sf_session.headers.update(...)

This should work fine within a view or something, since the auth/connection happens at the beginning of the Django run by default. There's some experimental and edge features in django-salesforce (like lazy loading and dynamic authentication) that probably would clobber this header, so that's worth remembering if your usage becomes more sophisticated.

This is exactly what "dynamic authentication" does:
connections['salesforce'].sf_session.auth.dynamic_start(access_token, instance_url=instance_url)
# some work...
connections['salesforce'].sf_session.auth.dynamic_end()
# forgot the access token and optionally use the auth values that were valid before .dynamic_start(...)

It is regularly tested by tests/test_dynamic_auth/tests.py.
It should be thread safe to use different tokens/organizations in different in individual threads. That was tested manually four years ago, but no automated test exists.

Three years ago I wrote also a modified general SalesforceAuth that can work with a custom authentication class configurable by DATABASES settings, not on only the default SalesforcePasswordAuth or a direct work with tokens by auth.dynamic_start(), e.g. a class for interactive authentication by "Force.com CLI" by web browser, useful for development with Salesforce, without saving a password anywhere. It is not a part of production repository still and it is little obsoleted because SFDX is probably a better option now. If that solution would be committed then the dynamic auth probably will have to be enabled in settings to work.

Thanks @philchristensen and @hynekcer, that does trick.

authors: Blekely hynekcer philchristensen

repo: django-salesforce/django-salesforce


text:
Fix es_MX translation on admin's change password

The hardcoded href produces an invalid link on 1.9+

Thanks for the report, however, translations are handled at Transifex.

repo: django/django


text:
Fixed wording of AuthViewsTestCase's docstring.

This fixes and improves the wording of AuthViewsTestCase's docstring.

@cjerdonek Thanks :+1:

authors: cjerdonek felixxm


text:
Fixed #35554, Refs #35060 -- Improved *args parsing in Model.save()/asave().

Trac ticket number
ticket-35554
Branch description
Change the parsing introduced in 3915d4c70d0d7673abe675525b58117a5099afd3 / PR #17667 to handle some edge cases, with new tests.
Checklist

[x] This PR targets the main branch.
[x] The commit message is written in past tense, mentions the ticket number, and ends with a period.
[x] I have checked the "Has patch" ticket flag in the Trac system.
[x] I have added or updated relevant tests.
[x] I have added or updated relevant docs, including release notes if applicable.
[x] I have attached screenshots in both light and dark modes for any UI changes.

@salvo-polizzi @jacobtylerwalls @felixxm : as you were involved in the original PR, would you like to take a look?

I also think we should fail if both args and kwargs were given for the same option, but perhaps this needs to be a separated ticket. Thoughts?

Yes, it would be good to protect against that case too. Maybe another ticket good just to keep this change focused.
The first pattern that comes to my mind is to check each variable against its default before picking from args:
if force_insert is not False:
    # uh oh
force_insert = args[0]
if total_len_args >= 3:
    if force_update is not False:
        # uh oh
    force_update = args[1]
...

A bit involved.

I think the branch is ready for landing, will mark it as so and merge when CI is green.
Thank you @adamchainz and @jacobtylerwalls! :star2

Thanks for the extra test and fixups @nessita.

Would you like to create the ticket and I'll triage it?

Done: https://code.djangoproject.com/ticket/35561

I also think we should fail if both args and kwargs were given for the same option, but perhaps this needs to be a separated ticket. Thoughts?

Yes, it would be good to protect against that case too. Maybe another ticket good just to keep this change focused.
The first pattern that comes to my mind is to check each variable against its default before picking from args:
if force_insert is not False:
    # uh oh
force_insert = args[0]
if total_len_args >= 3:
    if force_update is not False:
        # uh oh
    force_update = args[1]
...

A bit involved.

@adamchainz I have proposed #18313 to make the *args and **kwargs more strict and less repetitive. Please let me know what you think! :bow:

authors: adamchainz nessita


text:
Fixed #24230: Added translated name to language info dictionary.

See https://code.djangoproject.com/ticket/24230 for more info.

Current patch still misses docs and tests for filter.

Did you evaluate the solution of producing name_translated dynamically in get_language_info? The current proposal looks a bit needlessly repetitive in LANG_INFO.

That's a good idea. Can't remember why I chose the that solution. Patch updated.

Seems something is still missing...

I was too hasty... Will fix that tomorrow.

Tomáš, in the ticket you mentioned documentation... do you plan to add that soon?

Definitely, I'll do it today.

Rebase to latest master, moved code to get_language_info method so it can be used anywhere, added tests, updated documentation.

Don't forget to update the flags on the ticket so it appears in the review queue after you update it, thanks!

buildbot, test this please.

Add 1.9 release notes?

Added release notes. However I wasn't able to add cross-references because all these tags (get_current_language, ...) and filters (language_*) are all listed in one section and doesn't have associated labels.

We can fix the linking issue. Created https://github.com/django/django/pull/4737.

merged in 002b3d87b5f1e35edcb6e14ce2e2827fc6c032a1, thanks!

authors: MarkusH claudep timgraham tricoder42


text:
Fixed #23173 -- Fixed incorrect stripping of SCRIPT_URL

Fixed incorrect stripping of SCRIPT_URL

buildbot, test this please.

LGTM

:shipit:

Merged in 336512fae77ec214ad6db24166b9a8676007cc09. Thanks.

authors: MarkusH bmispelon bpeschier


text:
Fixed #24836 -- Made force_text() resolve lazy objects.

https://code.djangoproject.com/ticket/24836

:+1:

authors: carljm timgraham


text:
Fixed #20468 -- Added loaddata --exclude option.

Original patch by Alex Morozov.
https://code.djangoproject.com/ticket/20468

merged in ae2a7da86bf841d42be86dea3effb0261187c950, thanks!

authors: berkerpeksag timgraham


text:
Fixed #27834 -- Added StrIndex database function.

https://code.djangoproject.com/ticket/27834

Looks mostly good to me. A few questions.

What happens if the first argument on mysql is some kind of multibyte unicode and so is the search string? Does the binary have any effect there?

Have you tested with TextField? I guess there's no support from MySQL or Oracle since they use LOBs. Have we documented TextField limitations with other string based db functions? If we haven't already, then it's fine to ignore for this patch.

I added a test with TextField, looks good on those DBs. Will investigate the multibyte / unicode thing this evening.

In the Oracle database we have five different functions INSTR, INSTRB, INSTRC, INSTR2, INSTR4 that search a string for a substring. Maybe we should add test for unicode strings  e.g.
    def test_strindex_unicode(self):
        Author.objects.create(name='ツリー')
        Author.objects.create(name='皇帝')
        Author.objects.create(name='皇帝 ツリー')
        authors = Author.objects.annotate(fullstop=StrIndex('name', 'リ'))
        self.assertQuerysetEqual(
            authors.order_by('name'),
            [2, 0, 5],
            lambda a: a.fullstop
        )

to make sure that everything works fine.

According to this this bug, MySQL INSTR() won't allow accurate case-sensitivity and multibyte string comparisons. Just added the unicode test and I'm assuming it'll fail on MySQL. Any ideas? Or do I just add a warning in the docs?

@bradmerlin what are the options here for mysql? Perhaps we can land on the least worst solution. @adamchainz any mysql insights?

Without any new information, we have two choices: either don't support case sensitivity, or don't support multibyte strings. Supporting multibyte strings strikes me as the more important of the two.

Afraid I don't know any magic workarounds.
I'd be in favour of supporting multibyte strings over case sensitivity, MySQL is case insensitive by default anyway.

I added a commit with some edits, could you check it?

From https://docs.djangoproject.com/en/dev/ref/models/expressions/#django.db.models.Func:

Positional arguments can be expressions or Python values. Strings are assumed to be column references and will be wrapped in F() expressions while other values will be wrapped in Value() expressions.

In the current implementationStrIndex handles the second arg differently (it's wrapped in Value, but not F), isn't the mentioned behavior implied for all Func subclasses?

Can the second argument of the INSTR SQL function be a column name? If so, I guess it may make sense to make the change you suggest.

Yes it can, that's standard SQL. Makes sense to me to make the change for consistency, though it makes the most common case a little annoying because one has to write InStr('field_name', Value('needle'))

FWIW I think that if an API makes more sense without defaulting strings to F expressions, then that's fine, as long as the difference is documented in some way. Strings are assumed to be F expressions only because that's what the common case usually calls for, and it was backwards compatible.
I wonder if it would be worth defining a class property that _parse_expressions uses to wrap non-expressions, rather than having subclasses all implementing the same pattern over and over. It'd also let us document overrides in a standard way.

FWIW I think that if an API makes more sense without defaulting strings to F expressions, then that's fine, as long as the difference is documented in some way.

@jarshwah the thing I dislike about this approach is that users have to hit the documentation/source to figure out if it was decided a specific API defaulted to wrapping strings in F() or not. Being consistent about the need to wrap strings values in V() makes it a better user experience IMHO.

I'm in favour of being consistent too.

authors: adamchainz bradmerlin charettes felixxm jarshwah sir-sigurd timgraham


text:
Draft 17 header protection and key derivation

Fixes #93

Fixes #93

This currently nukes the PNE/header crypto test vector unit tests for lack of a convenient updated set of test vectors. If I can't find a set from another implementer I'll just generate some new ones manually.

I haven't been able to find a set of test vectors for header protection that I trust, so I added a basic self-test to fill in the gap in the mean time. No other outstanding issues that I know of.

repo: djc/quinn


text:
Implement TUOMValue record with class operators

The idea is to have a single type TUOMValue which can be implicitly cast between itself and a Double or String which carries unit-of-measure details with it. Internally, FValue shall remain the base value, while the converted value is accessed through other methods.
Furthermore, it should support math operators, which respect their selected UOM and calculate on a given base unit. For example, "4 Feet" + "2 Meters" - "14 Millimeters".

Implemented but untested. Need to establish list of all possible test scenarios.


text:
🛑 Writeguard is down

In f671837, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in ec66751.


text:
🛑 Writeguard is down

In 0ae6410, Writeguard (https://www.writeguard.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Writeguard is back up in 2c6e0b0.


text:
Translating plural statements, lack of documentation

Hello!
So far I have not been able to translate plural statements like for example:
$t('{count} car ::: {count} cars', {count: count},count)
How should my key-value pair look in the translation variable? I can't seem to find any information on this in the documentation. This is how my translation variable from English to Danish would initially look by looking at the docs, but nothing is being translated:
const da = {
  "{count} cars" : "{count} biler",
  "{count} car" : "{count} bil",
}

Hi @tikiatua
Is it possible to add the 0 option ? Meaning having apple: [ 'no apples', 'one apple', '{count} apples' ] ?
Thank you

authors: The-BlackWidow jonasgroendahl


text:
Data Transformation

Apply Data Preprocessing and feature engineering on dataset

[x] Create Preprocessor Object
[x]  Apply Scaling on numerical features
[x]  Apply OneHotEncoding on categorical features
[x]  Create Pipeline
[x]  Apply Pipeline On Both train and test dataset
[x]  Save Transformed data
[x]  Save Preprocessor Object

data transformation pipeline tested successfully


text:
🛑 Rampa_Blato is down

In 1045a1f, Rampa_Blato (http://rampa.ddns.net:8001/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Rampa_Blato is back up in 51e8d41 after 11 minutes.


text:
Remove or move path customization part of lesson 1

IMHO, this first section starts the workshop on a confusing foot since it doesn't work in all environments/shells and I recommend removing or moving it.

First things first! Your prompt (where you will enter Bash commands) can be confusing. To change it to the file path of your working directory followed by a dollar sign, type or copy/paste the below code:

Type or copy/paste PS1="\w\$ " into your prompt and press Enter.

Then, type exec bash into your prompt and press Enter.

Now, you will have a record of your location as you move around your computer!

NOTE: to display only the current working directoy - not the path - use a capital letter W in step 1 above instead :)

This was done in a previous commit.

authors: pattyf pssachdeva

repo: dlab-berkeley/Bash-Git


text:
Review of testing code

@yangraymond leads us in a tour of his testing code.

[ ] Figure out how to view raw TAQ file
[ ] Match SAS code results with Python
[ ] Make DATA_FILES actually a list

repo: dlab-projects/python-taq


text:
[RFC] Make std.algorithm.searching.skipOver an eponymous template to allow partial instantiation

From https://github.com/dlang/phobos/pull/5497#issuecomment-310282326

Argh! find does not use the eponymous template pattern, which means it's not partially instantiatable with just a pred.

It turns out that many functions in std.algorithm don't, so this is a RFC for evaluating the best way to move forward here. I picked the non-trivial function skipOver as an example.

I hate this pattern too. Can't we just add another template "overload" that only takes a predicate and then forwards to another version of skipOver?

I hate this pattern too. Can't we just add another template "overload" that only takes a predicate and then forwards to another version of skipOver?

How would this work without duplicating all function headers of skipOver?

Yeah, true. Looks like it can't be helped.

But seeing template skipOver is already a sign that something has gotten too complicated...

Err, isn't this the by-the-book standard eponymous template pattern which allows partial instantiation and is already used in lots of places in Phobos, such as map?

Btw another benefit of partial instantiation is that we get Dependency-Carrying Declarations a la DIP1005 for free :)
I wonder why DIP1005 didn't talk about this.
Consider a simple example:
template foo(alias pred = "a == b")
{
    import std.range.primitives : isInputRange;
    auto foo(R)(R r)
    if (isInputRange!R)
    {
        import std.algorithm.iteration : sum;
        return r.sum;
    }
}

void main()
{
    foo([1]); // <- remove this and no modules are imported
}

> rdmd -v foo.d
predefs   DigitalMars Posix linux ELFv1 LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 CRuntime_Glibc D_LP64 D_PIC assert D_HardFloat
binary    /usr/bin/dmd
version   v2.074.0
config    /etc/dmd.conf
parse     foo
importall foo
import    object	(/usr/include/dlang/dmd/object.d)
semantic  foo
entry     main      	foo.d
semantic2 foo
semantic3 foo
import    std.range.primitives	(/usr/include/dlang/dmd/std/range/primitives.d)
import    std.traits	(/usr/include/dlang/dmd/std/traits.d)
import    std.typetuple	(/usr/include/dlang/dmd/std/typetuple.d)
import    std.meta	(/usr/include/dlang/dmd/std/meta.d)
import    std.algorithm.iteration	(/usr/include/dlang/dmd/std/algorithm/iteration.d)
import    std.functional	(/usr/include/dlang/dmd/std/functional.d)
import    std.ascii	(/usr/include/dlang/dmd/std/ascii.d)
import    std.algorithm.internal	(/usr/include/dlang/dmd/std/algorithm/internal.d)
code      foo
function  D main
function  std.range.primitives.front!int.front
function  foo.foo!().foo!(int[]).foo
function  std.algorithm.iteration.sum!(int[]).sum
function  std.algorithm.iteration.sum!(int[], int).sum
function  std.algorithm.iteration.reduce!"a + b".reduce!(int, int[]).reduce
function  std.algorithm.iteration.reduce!"a + b".reducePreImpl!(int[], int).reducePreImpl
function  std.algorithm.iteration.reduce!"a + b".reduceImpl!(false, int[], int).reduceImpl
function  std.functional.F!("a + b", "a", "b").binaryFun!(int, int).binaryFun
cc /tmp/.rdmd-1000/rdmd-foo.d-14302BAA00C11879E03D4743294EEEA6/objs/foo.o -o /tmp/.rdmd-1000/rdmd-foo.d-14302BAA00C11879E03D4743294EEEA6/foo.tmp -m64 -L/usr/lib -Xlinker --export-dynamic -Xlinker -Bstatic -lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt -ldl

Now without the use of the eponymous template:
> rdmd -v foo.d
predefs   DigitalMars Posix linux ELFv1 LittleEndian D_Version2 all D_SIMD D_InlineAsm_X86_64 X86_64 CRuntime_Glibc D_LP64 D_PIC assert D_HardFloat
binary    /usr/bin/dmd
version   v2.074.0
config    /etc/dmd.conf
parse     foo
importall foo
import    object	(/usr/include/dlang/dmd/object.d)
semantic  foo
entry     main      	foo.d
semantic2 foo
semantic3 foo
code      foo
function  D main
cc /tmp/.rdmd-1000/rdmd-foo.d-14302BAA00C11879E03D4743294EEEA6/objs/foo.o -o /tmp/.rdmd-1000/rdmd-foo.d-14302BAA00C11879E03D4743294EEEA6/foo.tmp -m64 -L/usr/lib -Xlinker --export-dynamic -Xlinker -Bstatic -lphobos2 -Xlinker -Bdynamic -lpthread -lm -lrt -ldl

@andralex so what's your opinion on this? Can we move forward?

So the current crop of skipOver overloads is:
bool skipOver(R1, R2)(ref R1 r1, R2 r2)
if (isForwardRange!R1
	&& isInputRange!R2
    && is(typeof(r1.front == r2.front)));

bool skipOver(alias pred, R1, R2)(ref R1 r1, R2 r2)
if (isForwardRange!R1
	&& isInputRange!R2
    && is(typeof(binaryFun!pred(r1.front, r2.front))));

bool skipOver(alias pred, R)(ref R r1)
if (isForwardRange!R
	&& ifTestable!(typeof(r1.front), unaryFun!pred));

bool skipOver(R, E)(ref R r, E e)
if (isInputRange!R && is(typeof(r.front == e) : bool));

bool skipOver(alias pred, R, E)(ref R r, E e)
if (is(typeof(binaryFun!pred(r.front, e))) && isInputRange!R);

My first notes independent on this PR:

There are a few mistakes - the type of == and predicate must be if-testable. Only one overload gets that right!
It seems the constraint on the third overload is too tight - the range can be input, not forward
There are a few forms of duplication in the constraints:
** equality or predicates should always work the same
** in the one-range version the range must be input
** in the two-range version the first must be forward and the second must be input

Based on the above I'd say this PR may be worth it even if we discount for the advantages of partial instantiation and import.
I'll now make a pass through the code looking for details.

ping @wilzbach this is approved with a couple of nits, feel free to amend or commit as is

ping @wilzbach this is approved with a couple of nits, feel free to amend or commit as is

Thanks for the ping and I am well aware of it, but I have been pretty busy with life for the last days :/
Though I hopefully will get back to this soon :)

this is approved with a couple of nits, feel free to amend or commit as is

Okay I put this on the auto-merge queue now. IfTestable is only used in Phobos 10 times, so imho it's better to do these changes in a separate PR.

I turned off automerge because it was turned on for 6 days and did not merge due to failure(s) in CyberShadow/DAutoTest

ping @wilzbach

Merging manually as there seems to be something wrong with the CI status report from the auto-tester (probably it has sent more than 1000 status updates):

It's 100% green:

authors: CyberShadow MetaLang WalterBright andralex wilzbach

repo: dlang/phobos


text:
上传图片我设置sizeType只能是原图后，图片如果太大（2M）,会显示不出来

`
wx.chooseImage({
count: 1, // 默认9
sizeType: ['original'], // 可以指定是原图还是压缩图，默认二者都有
sourceType: ['album', 'camera'], // 可以指定来源是相册还是相机，默认二者都有
success (res) {
const src = res.tempFilePaths[0]
//  获取裁剪图片资源后，给data添加src属性及其值
self.wecropper.pushOrign(src)
}
})
`

#2

repo: dlhandsome/we-cropper


text:
Support for passing along options.data to node-sass

I have a use case where I need to be able to pass along the option data to node-sass. This PR fixes this.

Hmm, those failing tests seems to not be related to this PR. The tests pass when node-sass v3.4.1 is used (as specified in package.json) but not in the latest v3.4.2.

I'm unclear what the usecase is here. Please elaborate.

I want to be able to use the data option that exists in node-sass, but through the gulp-sass plugin.
I think the unit test I added shows the use case pretty good; to be able to pass arbitrary strings from JS to Sass.

This PR would be great for us. We would like to be able to pass our static asset path (as defined in our case by a django instance) to sass. In our case we cannot use relative paths as the CSS is being inlined into the HTML after being compiled to CSS (for performance and IE9 file size issue reasons).

This feature has been discussed at length and reject until a valid use case can be proposed. Please read the discussion in https://github.com/dlmanning/gulp-sass/issues/302 for more context.
If you believe you still believe you have use cases that doesn't violate the gulp semantics please leave a thorough description on that #302 and we'll reopen the discussion.

authors: jonathanp jonnyscholes xzyfer

repo: dlmanning/gulp-sass


text:
What is "Host, Username and Passwort"

sorry, but i just don't get it. What should be entered for host, username and password?
No username is required to access the KM2 via IP address, only a password.  So when I enter the IP of the KM2 as the host, I'm missing the username. the access data from Vbus.net are also not accepted.
Can someone help with what data I need to enter, please?

Try admin as user

See https://github.com/dm82m/hass-Deltasol-KM2/issues/18

wow, thanks for the incredibly quick response.  I would never have thought of "admin", but now it works.  Thank you very much.


text:
javascript.pegjs line 633...

          type:      "FuctionCall",

I think you meant
          type:      "FunctionCall",

@msackman Thanks for catching this!

authors: dmajda msackman

repo: dmajda/pegjs


text:
[Question] Chunking keys within a server

Hi! I have a question regarding accuracy of chunking a large key into a smaller key within a single server. I know MXNET_KVSTORE_BIGBOUND_ARRAY is used to chunk a key to different servers for processing.
Our implementation now chunks a physical key into a series of virtual keys, and these virtual keys are sent individually to one central server for optimization. The server only sees the virtual keys. We still use the vars for the original physical keys to synchronize access to the buffers. The virtual keys use buffers of the original physical keys (we chunk the buffer and assign different segments to the virtual keys).
We then send these virtual keys directly to optimize. For example:
update(self, index, weight, grad, state):
In here, index is a virtual key, instead of the original key.
Our problem is in a 0.3% accuracy drop with MNIST. In the first epoch, chunking vs not chunking's train accuracy starts to differs in 100-150 batches, but seems to be identical in the first 100 batches.  We do not know what may cause the problem, but it would be great if you can answer the following questions:

Is it fine to chunk keys and send each segments individually to server for optimization in general?
Can this be related to the optimizer? The optimizer is setup with states with physical keys, and it does not understand virtual keys, so it assigns default state for the keys it doesn't know?
Can this be related to update in per-key parameters? For example, learning rate for individual parameter (it only updates the learning rate for parameter it understands i.e. physical keys but not virtual keys)?

Many thanks!

After some experiments...
1, yes
2. yes
3. yes
:|

This issue is closed due to lack of activity in the last 90 days. Feel free to ping me to reopen if this is still an active issue. Thanks!

authors: Luo-Liang szha

repo: dmlc/mxnet


text:
[WIP] ndarray type support

This is a preliminary work for ndarray type support. It's not ready for merge. I'm posting this PR to solicit feedback. The corresponding mshadow PR: https://github.com/dmlc/mshadow/pull/89/files

[x] NDArray class type support.
[x] NDArrayFunction type support.
[ ] NDArray Unary Function type support.
[ ] Python ndarray interface type support.
[ ] Symbolic type inference.
[ ] Go over operators and implement type support.

I'll probably need to extend the c_api interface for NDArray when I implement python side. This probably will break all other front-end. Should we merge PRs and push together?
I added MX***Ex functions for dtype supporting interface.
This is a pretty big project. I can really use some help.
@tqchen Could you have a look?

Is this backward compatible? Please keep mxnet-scala team informed on major changes and possible breaks. @yanqingmen @javelinjs

Is this backward compatible? Please keep mxnet-scala team informed on major changes and possible breaks. @yanqingmen @javelinjs

Yes. everything is float32 by default

Yes. everything is float32 by default

moved to https://github.com/dmlc/mxnet/pull/1184

authors: piiswrong terrytangyuan


text:
Add norm_clipping and summary to module.

Add nd.global_norm and sym.global_norm
Add mod.clip_by_global_norm(max_norm=...)
Add mod.summary(level=...)
Add norm clipping to the RNN examples
The usage is
mod.forward_bacward(...)
mod.clip_by_global_norm(...)
mod.update()

The name of global_norm, clip_by_global_norm follows tensorflow (https://www.tensorflow.org/api_docs/python/tf/global_norm). And the summary follows keras.
The log of running cudnn_lstm_bucketing.py
WARNING: discarded 0 sentences longer than the largest bucket.
WARNING: discarded 0 sentences longer than the largest bucket.
WARNING: discarded 0 sentences longer than the largest bucket.
2017-04-16 18:27:02,240 Data:
2017-04-16 18:27:02,240    data: (85, 32), 2720
2017-04-16 18:27:02,241 Params:
2017-04-16 18:27:02,241    embed_weight: (10000, 200), 2000000, req = write
2017-04-16 18:27:02,241    lstm_parameters: (643200,), 643200, req = write
2017-04-16 18:27:02,241    pred_weight: (10000, 200), 2000000, req = write
2017-04-16 18:27:02,241    pred_bias: (10000,), 10000, req = write
2017-04-16 18:27:02,241 Aux States: None
2017-04-16 18:27:02,241 Total Param Num (exclude fixed ones): 4653200
2017-04-16 18:27:02,242 Total Fixed Param Num: 0
2017-04-16 18:27:02,242 Total Aux Param Num: 0
2017-04-16 18:27:02,242 Already bound, ignoring bind()
2017-04-16 18:27:02,242 optimizer already initialized, ignoring.
2017-04-16 18:27:03,534 Epoch[0] Batch [50]     Speed: 2344.07 samples/sec
Train-Perplexity=31611031.156206
2017-04-16 18:27:04,184 Epoch[0] Batch [100]    Speed: 2462.54 samples/sec
Train-Perplexity=6431.935574
2017-04-16 18:27:04,944 Epoch[0] Batch [150]    Speed: 2104.33 samples/sec
Train-Perplexity=1356.168854
2017-04-16 18:27:05,562 Epoch[0] Batch [200]    Speed: 2588.53 samples/sec
Train-Perplexity=1087.850005
2017-04-16 18:27:06,151 Epoch[0] Batch [250]    Speed: 2716.99 samples/sec
Train-Perplexity=1006.468296
2017-04-16 18:27:06,804 Epoch[0] Batch [300]    Speed: 2450.49 samples/sec
Train-Perplexity=923.301520
2017-04-16 18:27:07,456 Epoch[0] Batch [350]    Speed: 2454.61 samples/sec
Train-Perplexity=860.213869
2017-04-16 18:27:08,158 Epoch[0] Batch [400]    Speed: 2278.51 samples/sec
Train-Perplexity=774.597794
2017-04-16 18:27:08,773 Epoch[0] Batch [450]    Speed: 2601.98 samples/sec
Train-Perplexity=745.054633
2017-04-16 18:27:09,580 Epoch[0] Batch [500]    Speed: 1984.01 samples/sec
Train-Perplexity=693.528633

This looks hacky. Since we have backward inference now you can symbolically flatten and concat all weights into one. Just need to write a new op that split out n weights from a 1d long tensor

@piiswrong Do you think that the high-level norm clipping API is okay?
That is, we use
mod.forward_bacward(...)
mod.clip_by_global_norm(...)
mod.update()

making it a method of module seems weird

Or I can put max_norm to the update function? In fact, TF and PyTorch adopt the similar strategy like the above.
TF example: https://github.com/tensorflow/models/blob/master/tutorials/rnn/ptb/ptb_word_lm.py#L177
Pytorch example: https://github.com/pytorch/examples/blob/master/word_language_model/main.py#L139

One way of removing this from module would be to add a update_all method to optimizer that clip the gradient.

I've added lstm_ptb.py (Refer to PyTorch). @zihaolucky @mli @tangjiasheng
The running log:
(C:\Anaconda3) D:\HKUST\mxnet\example\rnn>python lstm_ptb.py --gpus 0
Vocabulary size = 10000
Training data: data\ptb.train.txt, number of words = 929589
Testing data: data\ptb.test.txt, number of words = 82430
Validation data: data\ptb.valid.txt, number of words = 73760
2017-04-18 02:51:50,442 Data:
2017-04-18 02:51:50,442    data: (35, 20), 700
2017-04-18 02:51:50,443 State:
2017-04-18 02:51:50,443    lstm0_begin_state_0: (1, 20, 200), 4000
2017-04-18 02:51:50,444    lstm0_begin_state_1: (1, 20, 200), 4000
2017-04-18 02:51:50,444    lstm1_begin_state_0: (1, 20, 200), 4000
2017-04-18 02:51:50,444    lstm1_begin_state_1: (1, 20, 200), 4000
2017-04-18 02:51:50,444 Params:
2017-04-18 02:51:50,444    embed_weight: (10000, 200), 2000000, req = write
2017-04-18 02:51:50,444    lstm0_parameters: (321600,), 321600, req = write
2017-04-18 02:51:50,445    lstm1_parameters: (321600,), 321600, req = write
2017-04-18 02:51:50,445    logits_weight: (10000, 200), 2000000, req = write
2017-04-18 02:51:50,445    logits_bias: (10000,), 10000, req = write
2017-04-18 02:51:50,445 Aux States: None
2017-04-18 02:51:50,445 Total Param Num (exclude fixed ones): 4653200
2017-04-18 02:51:50,445 Total Fixed Param Num: 0
2017-04-18 02:51:50,445 Total Aux Param Num: 0
curr_states: [[<NDArray 1x20x200 @gpu(0)>], [<NDArray 1x20x200 @gpu(0)>], [<NDArray 1x20x200 @gpu(0)>], [<NDArray 1x20x200 @gpu(0)>]]
d:\hkust\mxnet\python\mxnet\ndarray.py:817: RuntimeWarning: You are attempting to copy an array to itself
  warnings.warn('You are attempting to copy an array to itself', RuntimeWarning)
2017-04-18 02:51:53,634 Epoch:[0], Batch: [200]/[1327], lr: 1.000000, nll: 6.91065, ppl: 1002.9, grad_norm: 4.86222, ms/batch: 15.93
2017-04-18 02:51:56,241 Epoch:[0], Batch: [400]/[1327], lr: 1.000000, nll: 6.26552, ppl: 526.116, grad_norm: 7.17709, ms/batch: 13.03
2017-04-18 02:51:58,858 Epoch:[0], Batch: [600]/[1327], lr: 1.000000, nll: 6.01136, ppl: 408.037, grad_norm: 5.5614, ms/batch: 13.08
2017-04-18 02:52:01,456 Epoch:[0], Batch: [800]/[1327], lr: 1.000000, nll: 5.79843, ppl: 329.781, grad_norm: 6.22876, ms/batch: 12.99
2017-04-18 02:52:04,067 Epoch:[0], Batch: [1000]/[1327], lr: 1.000000, nll: 5.71415, ppl: 303.125, grad_norm: 5.99078, ms/batch: 13.06
2017-04-18 02:52:06,641 Epoch:[0], Batch: [1200]/[1327], lr: 1.000000, nll: 5.60375, ppl: 271.441, grad_norm: 6.28122, ms/batch: 12.87
2017-04-18 02:52:10,071 Epoch:[0], valid nll: 5.51714, valid ppl: 248.923
2017-04-18 02:52:10,071 Epoch:[0], test nll: 5.49152, test ppl: 242.625
2017-04-18 02:52:12,686 Epoch:[1], Batch: [200]/[1327], lr: 1.000000, nll: 5.56141, ppl: 260.188, grad_norm: 5.97247, ms/batch: 13.06
2017-04-18 02:52:15,296 Epoch:[1], Batch: [400]/[1327], lr: 1.000000, nll: 5.51113, ppl: 247.431, grad_norm: 7.18302, ms/batch: 13.05
2017-04-18 02:52:17,845 Epoch:[1], Batch: [600]/[1327], lr: 1.000000, nll: 5.48024, ppl: 239.905, grad_norm: 6.07934, ms/batch: 12.74
2017-04-18 02:52:20,490 Epoch:[1], Batch: [800]/[1327], lr: 1.000000, nll: 5.39276, ppl: 219.809, grad_norm: 6.65415, ms/batch: 13.22
2017-04-18 02:52:23,089 Epoch:[1], Batch: [1000]/[1327], lr: 1.000000, nll: 5.38767, ppl: 218.692, grad_norm: 6.0671, ms/batch: 13.00
2017-04-18 02:52:25,662 Epoch:[1], Batch: [1200]/[1327], lr: 1.000000, nll: 5.31094, ppl: 202.541, grad_norm: 6.1194, ms/batch: 12.86
2017-04-18 02:52:29,082 Epoch:[1], valid nll: 5.25973, valid ppl: 192.43
2017-04-18 02:52:29,083 Epoch:[1], test nll: 5.23009, test ppl: 186.81
2017-04-18 02:52:31,704 Epoch:[2], Batch: [200]/[1327], lr: 1.000000, nll: 5.32657, ppl: 205.731, grad_norm: 6.02341, ms/batch: 13.09
2017-04-18 02:52:34,262 Epoch:[2], Batch: [400]/[1327], lr: 1.000000, nll: 5.30369, ppl: 201.077, grad_norm: 7.06221, ms/batch: 12.79
2017-04-18 02:52:36,879 Epoch:[2], Batch: [600]/[1327], lr: 1.000000, nll: 5.29324, ppl: 198.988, grad_norm: 6.13586, ms/batch: 13.08
2017-04-18 02:52:39,469 Epoch:[2], Batch: [800]/[1327], lr: 1.000000, nll: 5.22072, ppl: 185.067, grad_norm: 6.38308, ms/batch: 12.95
2017-04-18 02:52:42,065 Epoch:[2], Batch: [1000]/[1327], lr: 1.000000, nll: 5.23997, ppl: 188.664, grad_norm: 6.14996, ms/batch: 12.97
2017-04-18 02:52:44,660 Epoch:[2], Batch: [1200]/[1327], lr: 1.000000, nll: 5.16351, ppl: 174.777, grad_norm: 6.22682, ms/batch: 12.98
2017-04-18 02:52:48,069 Epoch:[2], valid nll: 5.14275, valid ppl: 171.186
2017-04-18 02:52:48,069 Epoch:[2], test nll: 5.10692, test ppl: 165.16
2017-04-18 02:52:50,644 Epoch:[3], Batch: [200]/[1327], lr: 1.000000, nll: 5.1986, ppl: 181.019, grad_norm: 6.24097, ms/batch: 12.87
2017-04-18 02:52:53,268 Epoch:[3], Batch: [400]/[1327], lr: 1.000000, nll: 5.18811, ppl: 179.129, grad_norm: 7.24574, ms/batch: 13.12
2017-04-18 02:52:55,850 Epoch:[3], Batch: [600]/[1327], lr: 1.000000, nll: 5.18205, ppl: 178.048, grad_norm: 6.5865, ms/batch: 12.91
2017-04-18 02:52:58,447 Epoch:[3], Batch: [800]/[1327], lr: 1.000000, nll: 5.11657, ppl: 166.763, grad_norm: 6.61992, ms/batch: 12.99
2017-04-18 02:53:01,047 Epoch:[3], Batch: [1000]/[1327], lr: 1.000000, nll: 5.14813, ppl: 172.109, grad_norm: 6.41879, ms/batch: 13.00
2017-04-18 02:53:03,613 Epoch:[3], Batch: [1200]/[1327], lr: 1.000000, nll: 5.06755, ppl: 158.785, grad_norm: 6.41642, ms/batch: 12.83
2017-04-18 02:53:07,020 Epoch:[3], valid nll: 5.07307, valid ppl: 159.664
2017-04-18 02:53:07,020 Epoch:[3], test nll: 5.03242, test ppl: 153.303
2017-04-18 02:53:09,651 Epoch:[4], Batch: [200]/[1327], lr: 0.500000, nll: 5.07759, ppl: 160.387, grad_norm: 6.2272, ms/batch: 13.14
2017-04-18 02:53:12,223 Epoch:[4], Batch: [400]/[1327], lr: 0.500000, nll: 5.04949, ppl: 155.943, grad_norm: 6.77082, ms/batch: 12.86
2017-04-18 02:53:14,808 Epoch:[4], Batch: [600]/[1327], lr: 0.500000, nll: 5.03364, ppl: 153.491, grad_norm: 6.79625, ms/batch: 12.93
2017-04-18 02:53:17,398 Epoch:[4], Batch: [800]/[1327], lr: 0.500000, nll: 4.963, ppl: 143.022, grad_norm: 6.73801, ms/batch: 12.94
2017-04-18 02:53:19,985 Epoch:[4], Batch: [1000]/[1327], lr: 0.500000, nll: 4.9973, ppl: 148.012, grad_norm: 6.66943, ms/batch: 12.93
2017-04-18 02:53:22,590 Epoch:[4], Batch: [1200]/[1327], lr: 0.500000, nll: 4.90606, ppl: 135.106, grad_norm: 6.92826, ms/batch: 13.03
2017-04-18 02:53:26,001 Epoch:[4], valid nll: 4.94815, valid ppl: 140.914
2017-04-18 02:53:26,001 Epoch:[4], test nll: 4.90672, test ppl: 135.196
2017-04-18 02:53:28,605 Epoch:[5], Batch: [200]/[1327], lr: 0.500000, nll: 4.98666, ppl: 146.446, grad_norm: 6.60281, ms/batch: 13.01
2017-04-18 02:53:31,210 Epoch:[5], Batch: [400]/[1327], lr: 0.500000, nll: 4.97871, ppl: 145.287, grad_norm: 7.34425, ms/batch: 13.02
2017-04-18 02:53:33,798 Epoch:[5], Batch: [600]/[1327], lr: 0.500000, nll: 4.97228, ppl: 144.355, grad_norm: 7.05366, ms/batch: 12.94
2017-04-18 02:53:36,378 Epoch:[5], Batch: [800]/[1327], lr: 0.500000, nll: 4.90649, ppl: 135.164, grad_norm: 7.14363, ms/batch: 12.90
2017-04-18 02:53:38,997 Epoch:[5], Batch: [1000]/[1327], lr: 0.500000, nll: 4.94675, ppl: 140.716, grad_norm: 6.89783, ms/batch: 13.09
2017-04-18 02:53:41,540 Epoch:[5], Batch: [1200]/[1327], lr: 0.500000, nll: 4.86025, ppl: 129.056, grad_norm: 7.12102, ms/batch: 12.72
2017-04-18 02:53:44,945 Epoch:[5], valid nll: 4.91431, valid ppl: 136.225
2017-04-18 02:53:44,945 Epoch:[5], test nll: 4.87533, test ppl: 131.017
2017-04-18 02:53:47,576 Epoch:[6], Batch: [200]/[1327], lr: 0.500000, nll: 4.9385, ppl: 139.56, grad_norm: 6.59724, ms/batch: 13.15
2017-04-18 02:53:50,156 Epoch:[6], Batch: [400]/[1327], lr: 0.500000, nll: 4.93245, ppl: 138.719, grad_norm: 7.62306, ms/batch: 12.90
2017-04-18 02:53:52,751 Epoch:[6], Batch: [600]/[1327], lr: 0.500000, nll: 4.92692, ppl: 137.955, grad_norm: 7.1963, ms/batch: 12.97
2017-04-18 02:53:55,368 Epoch:[6], Batch: [800]/[1327], lr: 0.500000, nll: 4.87253, ppl: 130.652, grad_norm: 7.30981, ms/batch: 13.09
2017-04-18 02:53:57,934 Epoch:[6], Batch: [1000]/[1327], lr: 0.500000, nll: 4.91189, ppl: 135.896, grad_norm: 7.05739, ms/batch: 12.82
2017-04-18 02:54:00,573 Epoch:[6], Batch: [1200]/[1327], lr: 0.500000, nll: 4.82327, ppl: 124.371, grad_norm: 7.38883, ms/batch: 13.20
2017-04-18 02:54:03,995 Epoch:[6], valid nll: 4.88492, valid ppl: 132.28
2017-04-18 02:54:03,995 Epoch:[6], test nll: 4.84363, test ppl: 126.93
2017-04-18 02:54:06,583 Epoch:[7], Batch: [200]/[1327], lr: 0.500000, nll: 4.89303, ppl: 133.357, grad_norm: 6.96921, ms/batch: 12.94
2017-04-18 02:54:09,173 Epoch:[7], Batch: [400]/[1327], lr: 0.500000, nll: 4.89323, ppl: 133.383, grad_norm: 7.63543, ms/batch: 12.95
2017-04-18 02:54:11,773 Epoch:[7], Batch: [600]/[1327], lr: 0.500000, nll: 4.89055, ppl: 133.026, grad_norm: 7.51316, ms/batch: 13.00
2017-04-18 02:54:14,346 Epoch:[7], Batch: [800]/[1327], lr: 0.500000, nll: 4.82974, ppl: 125.179, grad_norm: 7.24595, ms/batch: 12.86
2017-04-18 02:54:16,943 Epoch:[7], Batch: [1000]/[1327], lr: 0.500000, nll: 4.8794, ppl: 131.552, grad_norm: 7.23011, ms/batch: 12.99
2017-04-18 02:54:19,503 Epoch:[7], Batch: [1200]/[1327], lr: 0.500000, nll: 4.78783, ppl: 120.041, grad_norm: 7.74472, ms/batch: 12.80
2017-04-18 02:54:22,927 Epoch:[7], valid nll: 4.86416, valid ppl: 129.562
2017-04-18 02:54:22,927 Epoch:[7], test nll: 4.82374, test ppl: 124.43
2017-04-18 02:54:25,546 Epoch:[8], Batch: [200]/[1327], lr: 0.250000, nll: 4.85531, ppl: 128.42, grad_norm: 7.08554, ms/batch: 13.09
2017-04-18 02:54:28,149 Epoch:[8], Batch: [400]/[1327], lr: 0.250000, nll: 4.83528, ppl: 125.874, grad_norm: 7.92807, ms/batch: 13.01
2017-04-18 02:54:30,735 Epoch:[8], Batch: [600]/[1327], lr: 0.250000, nll: 4.82842, ppl: 125.014, grad_norm: 7.50268, ms/batch: 12.93
2017-04-18 02:54:33,339 Epoch:[8], Batch: [800]/[1327], lr: 0.250000, nll: 4.75766, ppl: 116.473, grad_norm: 7.70178, ms/batch: 13.01
2017-04-18 02:54:35,893 Epoch:[8], Batch: [1000]/[1327], lr: 0.250000, nll: 4.80403, ppl: 122.002, grad_norm: 7.64826, ms/batch: 12.76
2017-04-18 02:54:38,511 Epoch:[8], Batch: [1200]/[1327], lr: 0.250000, nll: 4.70713, ppl: 110.734, grad_norm: 7.79127, ms/batch: 13.09
2017-04-18 02:54:41,916 Epoch:[8], valid nll: 4.82044, valid ppl: 124.02
2017-04-18 02:54:41,916 Epoch:[8], test nll: 4.7763, test ppl: 118.665
2017-04-18 02:54:44,503 Epoch:[9], Batch: [200]/[1327], lr: 0.250000, nll: 4.80568, ppl: 122.203, grad_norm: 7.37722, ms/batch: 12.93
2017-04-18 02:54:47,150 Epoch:[9], Batch: [400]/[1327], lr: 0.250000, nll: 4.80226, ppl: 121.786, grad_norm: 8.13728, ms/batch: 13.24

Also, I haven't tested it on multiple GPUs and there is a minor batch padding issue (i.e, the last few instances are not used).

I'm going to close the PR since there is no activity for more than 2 months. But please feel free to re-open it and update with the master.

@eric-haibin-lin
# Small Configuration
python lstm_ptb.py --num-hidden 200 \
                   --num-embed 200 \
                   --init-scale 0.1 \
                   --lr-decay 0.5 \
                   --decay-epoch-num 4 \
                   --dropout 0.5 \
                   --max-norm 5 \
                   --gpus 0
# Medium Configuration
python lstm_ptb.py --num-hidden 650 \
                   --num-embed 650 \
                   --init-scale 0.05 \
                   --lr-decay 0.8 \
                   --decay-epoch-num 6 \
                   --dropout 0.5 \
                   --max-norm 5 \
                   --gpus 0
# Large Configuration
python lstm_ptb.py --num-hidden 1500 \
                   --num-embed 1500 \
                   --init-scale 0.04 \
                   --lr-decay 0.87 \
                   --decay-epoch-num 14 \
                   --dropout 0.65 \
                   --max-norm 10 \
                   --gpus 0

authors: mli piiswrong sxjscience


text:
[Relay] Resolve cyclic dependency in Closure

Right now relay leak memory for all of the executor (aot, interpreter, vm), for even the simplest program (id = fun x => x). They all happened to be caused by the exact same problem.
In Relay Implementation, a closure capture over all the free variables, holding a reference to all of them. However, it also hold a reference of itself (as long as it has a name, while ANF give everything a name). This make all closure cyclic data structure.
I propose to solve it by making the two following modification:
add a new Value, RecFunc. RecFunc hold a shared_ptr of a vector of closure, and a size_t index into it.
Every Closure will now have another field, a Map<Var, size_t> which store all recursive call (if any, including mutually recursive one) from the parent RecFunc.
This is in inspiration from https://cs.indiana.edu/ftp/techreports/TR73.pdf -

I will submit a PR soon, issuing here for visibility so ppl can avoid the same problem again.

I am not sure I get it, as for recursive call we are capturing Variable, as opposed to the its binding I think this problem worth a bit of more thoughts. Essentially it was the same problem in the global modules, in which case we resolve by global var. Do you think if it is possible to do the same thing, eg lamda lift then only capture global vars. You will need the module map to be able to run closure

@MarisaKirisame given that it is an interesting design question, can you open an RFC discuss thread ?

let's take this standard program, the factorial, for example.
let fact = \x -> if x == 0 then 1 else x * fact (x - 1)
The closure object of x contain the value of all free variable of fact - in that case, also a shared_ptr reference to the fact closure too!
I will open an RFC later.

consolidate to #4143

authors: MarisaKirisame tqchen

repo: dmlc/tvm


text:
[Relay, OpFusion] Better tuple fusion implementation

See #3039 for the context and discussion.
This is my second cut at fixing tuple fusion, which I hope is a better approach than the rather ad hoc one in #3049 .

Added two new op patterns, for marking tuple and tuple field nodes
Added a new attribute to the fusion group structure, to keep track of data flow between groups (think hypergraph where nodes correspond to fusion groups)
After the existing fusion step, check each fusion group to see if we can further fuse the group and its input groups.  By now, we can determine If the tuple has been fused to subsequent ops or not. We can fuse tuple fields into the tuple here.

@tqchen I need to fix the tensorflow test, but can you have a look and let me know if I am on the right track?

I think we can still simplify it. We do not need to record the inputs of the group. We can still just do the traversal in the node relation group.
If there is a relation x -> tuple, and we can directly check if the group of the tuple is already injective, which means it is already fused to another op. Then we can fuse the group of x and group of tuple, if the group of the x is injective, and everything in between x and tuple follows the same pattern

thanks, I was also able to remove kTupleFields, one of two new op patterns I added. I think it is much simpler now.

@MarisaKirisame @vinx13 please help review this PR

Ready for review @tqchen @jroesch @MarisaKirisame @zhiics @vinx13

Given that there is a lot of recent interest in the fusor, I opened a new issue for better docs https://github.com/dmlc/tvm/issues/3109

LGTM

LGTM

@vinx13 @tqchen looks like CI is broken, I'm getting error from topi group conv2d test (verify_group_conv2d_NCHWc_int8). Maybe #3070 is related?
http://ci.tvm.ai:8080/blue/organizations/jenkins/tvm/detail/PR-3092/5/pipeline/235

Thanks @masahi @vinx13 @zhiics , this is merged

authors: masahi tqchen zhiics


text:
Reason to require R >= 3.3.0 ?

I have to use R 3.2 in my workplace. But latest xgboost R package 0.6-3 requires R >= 3.3.0
I see 0.6-2 was requiring >= 2.15.1, and there was a commit https://github.com/dmlc/xgboost/pull/1920/commits/d05744cd5a513dfb6922c7f23e7a0f75d2bc81bd mentioning bumping to R 3.1.0.
I wonder if there's any reason to require 3.3.0. If not can we get it down to 3.1.0 or 3.2.0? Thanks.

The requirement R >= 3.3.0 is basically to use the new Rtools with GCC 4.9.
The package can't compile using the old Rtools: https://www.r-project.org/nosvn/R.check/r-oldrel-windows-ix86+x86_64/xgboost-00check.html

I just replied in #1267 that some change in dmlc-core\include\dmlc\base.h https://github.com/dmlc/xgboost/issues/1267#issuecomment-231975899 helped to solve the building issue with R 3.2. Can we get that change into dmlc?

I don't have strong opinions on this. However, we used to have some problems in fopen64. Different tool chains might have different definition. See https://github.com/dmlc/dmlc-core/pull/124 and https://github.com/dmlc/dmlc-core/pull/143/ as reference.
Ideally we should test different tool chains.

If possible, I think it might be helpful to keep backward compatibility on g++4.6.
Just to make sure we don't break things. @thirdwing can you look into this?

This can be done if I have time. But it seems better to pass all CRAN check first.

@tqchen @hetong007 We can support old Rtools by removing these three lines https://github.com/dmlc/dmlc-core/blob/master/include/dmlc/base.h#L138-L140
But we need to test some more distribution of MinGW.

we do need to define fopen64 because it is used in later code, but maybe not with prefix std

I know what you mean.
My suggestion is that (1) we provide a recommended version of MinGW-w64 (for example, TDM distribution from http://tdm-gcc.tdragon.net/) and add it into our documents (https://github.com/dmlc/xgboost/blob/master/doc/build.md#building-on-windows); (2) besides this MinGW-w64, we also support the Rtools with GCC4.6.3 and GCC4.9.2.

Sounds good, I think Rtools compatibility is more important, and the other windows builds will be recommended to direct to CMake based builds

I think it is good enough if we guarrante xgboost can compile using two Rtools and one MinGW. I will let you know after testing.

I'm planning a new CRAN submission for R 3.4.0.
Currently we still require R 3.3.0, any progress on releasing the constraint?

Hi All,
I also have to use R 3.2.5 and thus cannot install xgboost. Any updates regarding this issue?
Thanks!

After making a new release, I will fix this.

Hello I am in the same boat, and I was wondering if there is a workaround to letting R 3.2 install xgboost?

As a workaround, you might try installing it from source where you would first modify the R-package/DESCRIPTION file to have the "R (>= 3.2.0)" dependency. You would still need to use a C++11 compatible compiler though.

Hey Khotilov, thanks for the quick response.
Based on your explanation I found an older commit and tried installing like this:
install_github('dmlc/xgboost',subdir='R-package', ref = "7948d1c")
However, it fails with the following warning message.
Warning message:
GitHub repo contains submodules, may not function as expected!
I've searched around and found thirdwing saying devtool doesn't support submodules, and found a long explanation on how to supposedly do this:
https://github.com/dmlc/xgboost/issues/776
They seem to be using cygwin to install from a github clone. I haven't yet tried this; however, in the end, I need to install it on a EC2 instance which is Linux so I'm very hesitant to try it out. I would very much appreciate it if anyone can provide me some more guidance on what is the best way to install an older version that works on R 3.2.
Thank you so much!

I would do a clone of the repo and build from there.
To get the submodules, please look at : https://github.com/dmlc/xgboost/blob/master/doc/build.md
Then you can easily do the modification described by @khotilov

repo: dmlc/xgboost


text:
[jvm-packages] How can a C++ program use a xgboost model trained by xgboost4j.

I wanted to train a xgboost model by xgboost4j-spark, and use the model for prediction in a C++ server program. But it failed due to the below code in the red circle.

The code is from https://github.com/dmlc/xgboost/blob/6f16f0ef5829965ba420ef6937617600d286820a/src/gbm/gbm.cc

how you saved the model?

I have the same problem，I used func saveModelAsHadoopFile

@CodingCat thank you for your response. I saved my model using the following code
if (save_path != ""){
model.saveModelAsHadoopFile(save_path)
}

saveModelAsHadoopFile can only be consumed by xgboost4j-spark, if you want to cross-languages consuming, you have to call https://github.com/dmlc/xgboost/blob/72451457120ac9d59573cf7580ccd2ad178ef908/jvm-packages/xgboost4j/src/main/scala/ml/dmlc/xgboost4j/scala/Booster.scala#L149

I am just faced the same problem and it works now.

First, I suggest that you could compile the xgboost as an dynamic library and use it online in your c++ server, which is what we do now.  Including the source code from xgboost in your server may be difficult.

Train model on Spark has no problem.  Just notice Spark does the internal conversion, and does not accept formats that are 0-based.  This is very important which can cause feature index mismatch.

source code of loadLibSVMFile in Spark is as bellow:
def loadLibSVMFile(
      sc: SparkContext,
      path: String,
      numFeatures: Int,
      minPartitions: Int): RDD[LabeledPoint] = {
    val parsed = sc.textFile(path, minPartitions)
      .map(_.trim)
      .filter(line => !(line.isEmpty || line.startsWith("#")))
      .map { line =>
        val items = line.split(' ')
        val label = items.head.toDouble
        val (indices, values) = items.tail.filter(_.nonEmpty).map { item =>
          val indexAndValue = item.split(':')
          val index = indexAndValue(0).toInt - 1  // Convert 1-based indices to 0-based.
          val value = indexAndValue(1).toDouble
          (index, value)
        }.unzip
        .......

@CodingCat thank you. I will have a try.

It works. Thank @CodingCat and @zmingshi

authors: CodingCat algorithmdog jichao6758 zmingshi