text:
Cache build information between webpack processes

The current watch mode for webpack is fairly good at caching load results and general workload between file changes as long as you keep the same webpack process running. The initial build for some projects, on the other hand, can be on the order of more than a minute. This can make the build process very slow if for some reason you need to stop your watcher, or node crashes, or you simply want to restart your connect server with different flags.
It would be nice if webpack were able to persist compilation information between invocations of the node process to the filesystem and reload that cache later -- so ideally, performance between different $ webpack calls would be close to how long it takes to --watch to recompile when it detects a change.

Running into this problem as well.  Once its going its instant, but right now our initial build takes over a minute to warm up.

This would be really cool.

This could be very useful also for speeding up incremental builds in a continuos deployment setup.
@sokra could you point us in the right direction if we want to help with this?

Any pointers on where to start to solve this issue?

I took a look at serialising the cache based on a TodoMVC project (babel, sass) that I have lying around and Webpack 1.12.0. Here is what I found.
The compiler.cache is provided by CachePlugin created by WebpackOptionsApply. In CachePlugin it is trivial to serialise on the existing "after-compile" step and reload the same file (if it exists) in the constructor. Sure it has circular dependencies but cycle may be used.
However this doesn't get us anywhere. The reason begin that the cache is not a plain data structure. It is composed of Classes that would need to be identified on serialisation and reconstructed on deserialisation. The deserialised object does not implement the requisite methods and the compiler quickly fails on cacheModule.needRebuild().
So how big is this problem? I hacked this code in the CachePlugin "after-compile" to see.
I found that all my interesting files (in the small Todo project) were entries in the cache hash each with value instanceof ContextModule. Noting that ContextModule inherits Module inherits DependenciesBlock which uses DependenciesBlockVariable.
Enumerable on these module instances there was also instances of Parser, OriginalSource, SourceMapSource (or Array thereof).
At first glance it is plausible (but suboptimal) to write an external monolithic serialiser/deserialiser that can reconstruct these objects. It would support only this set of Classes but not necessitate any changes to the Webpack codebase.
Likely such and implementation would only be useful for a narrow set of use cases. Given the large number of Classes in the Webpack /lib I imagine there would be a quickly growing list of other classes that would need to be included in the monolith.
The alternative is where @sokra brings the difficult tag - Much or all of the Webpack /lib needs to support some sort of serialisation interface, in order that the cache items may be recursively serialised/ deserialised. This would be the more consistent solution but needs architectural buy-in.
Please let me know if any of this is misinformed. I'm interested to hear other thoughts on this.
I would like to try the monolithic serialiser. Obviously as its author it would be suited to my use case, but also because it might inform a more general solution.
If you know of any other Classes that would need inclusion (or you can discover them by running a similar test) then please let me know. But my hope is that the full cache would not need to be reinstated to get some performance benefit.

Ok, so I have a really naive implementation of the monolithic serialisation. Overall... interesting but currently unusable.
There were lots of classes in the compiler.cache that I initially didn't see. But thankfully I was able to use the require.cache to discover pretty much every Class and @sokra unintentionally did an absolutely beautiful job of structuring all the classes so that they could be easily reconstituted.
Without optimisation serialisation is 2 orders of magnitude in the red, which doesn't bode well. Deserialisation is quite fast, leading me to believe it is the class discovery hurting the serialisation. In the short term I added a persist:false option to turn off serialisation and just use any existing cache file.
Even under these conditions performance gain is only 10% on a medium project. For very small projects this is completely offset by the time to deserialise the cache. At this stage its pretty amazing it produces workable output at all. But what we are all expecting is incremental-size compile times. So I have to conclude that right not it is not useful.
Some learning:

The cache is much more complex than I expected from my experience working with the Browserify cache. I am sure this is essential complexity but I'm hoping we can be selective in what we persist. This really needs some expert knowledge.

Possibly doesn't need an architectural solution due to the clean nature of @sokra's classes. Notwithstanding the possibility that the whole concept just doesn't work.

Currently writing JSON to disk. I'm seeing 200MB cache file on a medium size project with built size around 4MB. Binary will help but how much (?).

With lots of copies of Webpack code it was difficult to detect Classes from the correct paths, making require.cache essential. I need to re-confirm this wasn't something stupid I was doing during development which is now fixed. I'd like to require-dir on the Webpack /lib folder instead.

The small project mentioned above builds correctly but that is no guarantee for other use cases. I don't have any medium/large projects in a stable state (migrating from Browserify) but I suspect a problem with at least one. All are AngularJS projects. It would be good if others can test their use-case to see where things stand.

I've spent a couple of days on this and need to let it sit for a while. But if it crashes for you certainly raise an issue and I will try to look at it right away.
This is such a missing link it would be awesome to crack it. If anyone has feedback or PR that can move this plugin forward please let me know.

@bholloway Nice work and writeup! Apologies if this is obvious, but how much of the 200MB is duplicated? Are the same references being serialized multiple times? You might also be running into memory pressure/gc just from serializing/deserializing the entire thing into memory at once (as opposed to streaming/multiple cache files).

Running an amended version of Douglas Crawford's decycle to remove
duplicates and circular references.
Forgot to mention that the emit phase is running for reach compiler. Need
to find an async  phase for overall completion and act just once.

@bholloway pretty awsome work.
Nice idea of using require.cache for serialization. I initally thought about adding a static property to each class storing __filename of the module.
On of the reasons why the cache is so big is propably that modules are stored into the cache while still connected in the graph. They are disconnected when they are read from the cache (search from "disconnect" in the webpack source). I'll try to change this in webpack (may reduce memory usage too) and check how much effect does it have on the serialization.

I tried a similar approach as @bholloway and found the serialization/deserialization costs were too high, it's really nice to see that the problem wasn't as insurmountable as it appeared to me.
I ended up wrapping webpack, caching it's output, and adding a dozen or so lines of cache validation logic (checking mtimes on dependencies and emitted assets, package versions, etc). For our purposes (automated deploys with remote builds) this dropped webpack's build times from ~30 seconds to < 1 second - most of which is spent booting the node process, loading the modules and cache, and hitting the file system.
The same wrapper is used on a bunch of local dev machines as well. We serve from the cache, while spawning a watching compiler in the background. Once the compiler was ready, the cache would delegate to the compiler, and the compiler would keep the cache populated.

Probably repeating @bholloway points, but the blockers that I can remember encountering when trying to add a persistent cache to webpack were:

Cyclical structures in both the cache and the state
Mixtures of simple objects and instances with constructers+prototypes

The cyclical structures ensured that you had to walk the entire tree to find any cycles, which is slow on both ends of the serializer.
The objects with constructors and prototypes ensured that you would need a large amount of domain knowledge to cover all the use cases in both webpack and the ecosystem.
If those two issues were solved, performant caching would be pretty viable.
The last thing I remember considering was adding an interface for plugins and loaders to populate and read from the cache.

I've been looking at this too. Here's my current implementation. https://github.com/mzgoddard/webpack-cache-module-plugin
I dove into webpack's source while trying my hand at it so far. I realized the cyclical parts of the cache are caused by Compilation handling built modules and primarily storing Reasons that point to the module that depends on the newly built module. If a cached module is used the old Reasons are thrown away and new ones are computed during an iterative build.
From reading some other plugins I thought I might use this detail and hard code the members of modules to be serialized. To deserialize them I'd let webpack's normalModuleFactory do a bunch of the lifting and in the factory's module plugin interface wrap the NormalModule in a proxy object that uses the specifically selected serialized members for the first run if the timestamp wasn't out of date for its dependencies.
This works. It works for at least in my generated sandbox. I think the take away either way is to when serializing or deserializing the cache treat the modules as disconnected (https://github.com/webpack/webpack/blob/f7d799adb4dc52c7548cea63f5b08edfcf4476f6/lib/Module.js#L29-L37). As far as I can tell Module#disconnect removes the circular references and its essentially the state the modules are in when they are first created and the state Compilation wants when using a CachedModule.
The other thing I noticed, and I'm not sure if it affects @bholloway's work but I think it does, is even with using these cached proxies around the normal modules I saw maybe a 10% improvement in performance. It wasn't after some more poking around and logging timestamps that I noticed the biggest consumer of time was spent resolving locations of dependencies. Implementing a stored UnsafeCache (https://github.com/mzgoddard/webpack-cache-module-plugin/blob/0f09a55b01715c8e70dbc64098a220b6f5c67f68/lib/CachePlugin.js#L80-L97) gave me the performance improvement we're all hoping for from a persisting cache. Compilation when processing modules for dependencies and recursing, building the depended on modules, has to resolve what that dependency is, or more specifically the module factory has to. I thought I knew how the iterative builds avoided this but looking back at webpack's source I'm not as sure. Just thinking about it now I'd guess the CachedInputFileSystem is helping here once primed from the first run.
Freezing and thawing the cache to disk is pretty straight forward. I think caching dependency resolutions or possibly part of the CachedInputFileSystem is where a large win may be waiting but they are less straight forward. One thought is if we could list all attempted filepaths for context and dependency pair, we might be able to watch or check those paths on start to invalidate path resolutions. Another thought is maybe CachedInputFileSystem can be a help here. Maybe if we persist to disk the stats, readlink and readdir info in there and check it on revalidate that on start we could gain a similar impact to UnsafeCache but be ... safer. I imagine though that revalidating info for the CachedInputFileSystem could take a lot of time considering the number of paths involved in file resolving dependencies and loaders.
To reiterate, from what I've found, Reason and Chunk objects and others removed in disconnect stored on Modules are the circular references and are not wanted during iterative compilations so they can be safely ignored. Resolving dependencies is very elaborate and takes a lot of time, to approach iterative build times somehow safely caching dependency path resolutions or CachedInputFileSystem is likely a needed step.
Hope this helps.

@sokra Is there anything in webpack 2 that would make caching to disk easier?

TLDR: persistent caching gets a lot easier when it's low-level and granular.

I took some time over the end of year break to do a greenfield reimplementation of a subset of webpack's features. It's reasonably functional and I'm using it currently in dev environments on some personal and work things. I do want to clarify that this is not intended as a webpack replacement, it's foremost a research project that I used to investigate and experiment within the problem space of building for the web.
One of the problems that I focused on solving was persistent caching of build data. I went through a couple of iterations before settling on the current approach that balances various trade-offs while providing some significant performance improvements. In one case, a project with an initial build time averaging ~25 seconds drops to ~2 seconds.
Background
The persistent cache mechanism that's in there is novel for two reasons - the population and the invalidation mechanisms.
Cache data is populated by various jobs in the pipeline that annotate per-file objects with data that is persisted and - once the process reboots - then rehydrated on-demand as each file is run through the pipeline again. If cached data is available, jobs can simply return the annotation on the object, rather than recalculate anything. To prevent delaying the build process, cache writes occur after the build has been emitted.
Invalidation occurs at two levels: all data is namespaced during the process's boot and the pipeline contributes a unique cache key for each file. The namespace enables high-level invalidation of the cached data, it's mostly used to discard data when changes are made to build scripts, node_modules, package.json, etc. The cache key enables low-level invalidation which is contextual to the specific file itself and is composed of the path, mtime and (in some cases, to get around OS time accuracy issues) a hash of the textual content.
Lessons learnt

Storing all data in a single json file is (in some cases) the most performant approach, but it'll lock up your process for a while as JSON.parse or JSON.stringify generates stuff. With a single file, you also need to invalidate data more regularly otherwise the (de)seralization costs grows rapidly. We store each file's data in a distinct file, while there are some IO overheads that this introduces, it helps to reduce blocking the event loop and removes most of the need to invalidate data from the cache.
Invalidation is a pain and you'll never get it perfect, so make it easy to nuke the cache or turn it off. There are endless edge-cases that you can't reasonably detect.
Mock caches are useful for testing and profiling code-paths without having to deal with integration issues.
Leave the cache at an extremely low and granular level so that subsystems can override or ignore it. It's tempting to simply skip jobs if you have cached data that you think is probably correct, but there is a lot of domain knowledge in each part of a build tool and you'll want to enable each part to control how their data is handled. For example: we cache dependency resolution with varying degrees of aggression based on the location of a file, in some cases we want to cache everything, in other cases we only want to cache package dependencies - hence, the job still needs to run so that it can decide how much of the data it wants to reuse or recompute.
Only cache the most expensive data. Most of the data associated with the build can be regenerated in a fraction of a second and hence, there's no need to preserve it as it'll only add to the re-hydration overhead and overly complicate your codebase. Code generation, sourcemaps, and dependency resolution - these are the really expensive parts that you don't want to recompute unless necessary.
Don't cache large data structures such as ASTs, the (de)serialization cost is enormous. You'll only want to cache the final outputs, such as code strings, sourcemaps or dependency lists.

Musings/recommendations for webpack
A persistent caching system comparable to what we've implemented could be added to webpack reasonably easily. I'd strongly recommend making small, incremental steps with it and profiling code to determine the appropriate tasks to cache.
As a starting point, loaders might be an appropriate place. Assuming that the loader interface was extended to add support for caching, they could opt-in to persistent caching of their computed data via an interface like this.persistentCache() which accepts a callback or returns a promise resolving to the data. If either the cache or data is not available, they can compute as before, and if the cache is available, populate it before exiting.
By adding opt-in persistent caching support as a low-level functionality for specific parts of the process, build-time could be steadily reduced without adding too much complexity to webpack's core. By making it an opt-in process, it's backwards compatible and is a feature that newer versions of loaders or plugins can take advantage of.
With regards to cache-invalidation, it gets a little bit more complicated, but you could take a comparable approach to what we have taken, eg: high-level and low-level cache invalidation. A start for a high-level scan would be checks on the config file or a hash of the config object, combined with a superficial check on the root of the node_modules dep tree. I extracted the mechanism that we use for high-level scans as env-hash, if it's of any assistance. For low-level invalidation, the path, mtime and content hashes would probably be fine.

Thx @markfinger for sharing your insights. :+1:
I've mentioned your project in a SO answer.

The last few weeks I gave trying to cache webpack builds another look. Since my other post on this issue I've learned more about building complex webpack plugins. That lead me to my current approach which I think is feasible for at least the iterative development use case.
My current work is up on github and npm at https://github.com/mzgoddard/hard-source-webpack-plugin and https://www.npmjs.com/package/hard-source-webpack-plugin respectively.
This plugin doesn't try to freeze and thaw the cache that Compilation and CachePlugin operate on. Instead it stores the rendered source of a module in a hard non-mutable shape and its necessary metadata. It invalidates a cached hard source version of a module by not returning a hard source module and instead letting the normal build process occur.
My last disk caching attempt was incredibly fragile. Small missing unconsidered details led to broken builds. Working on that version I was constantly discovered unconsidered elements. I didn't feel confident about it and had no desire to maintain it.
Comparing my experience with hard source to my last attempt, its been drastically different. Every thing I have missed and fixed has been much easier to solve and the number of missed details has been very manageable. Most of the details relate to dependencies. While the hard source drops the use of dependencies for secondary renders of the module, the module still needs the dependencies so webpack can discover the module graph and construct the wrapping around a module that isn't stored in the hard source. What has made this manageable to me is reducing the set of dependencies to a smaller set of custom dependencies than needed to originally render the module.
The only commonality hard source holds with my last attempt is a resolve cache. Like last time I still find caching path resolutions to be important for performance gains. Its not as impactful as the last attempt since much more savings are made with the modules. But still very noticeable.
I hadn't checked this issue before taking this shot at caching so its really refreshing to see that my approach and plans mirrors points made by @markfinger. Like on invalidation. Definitely going to use env-hash for assisting in namespace invalidation.
The HardSource plugin also sees a magnitude difference in performance. I haven't been able to test on a large project yet, but testing on two smaller personal projects I've seen times for a ~5s project drop down to ~500ms and a ~8s project drop down to ~1s. Those projects respectively have ~600 and ~400 modules. I already know how to improve the plugin for that second case to drop it into a magnitude difference. (The difference for the latter is a lot of large binary assets that the plugin handles in a very inefficient format at the moment.) I haven't analyzed it enough but at this point I suspect the time left is spent building the module graph, "sealing" the compilation, rendering the chunks, and emitting assets. (Naturally I'm curious about investigating if the plugin can assist with building the module graph and emitting assets. The others too, I just figure those would be easier to bring about an improvement.)
I didn't expect to be so excited to share this and hope it can help lift some of the pain of long webpack build times. Hopefully this can help contribute to any further iterations to building webpack iteratively as well. :smile:

@mzgoddard that looks really cool. I didn't dig too deep into the source, but you're definitely showing a lot of experience with webpack and its internals.
Hmm, since my last post in here, I've continued tinkering and working on some ideas:-
Cache storage/retrieval
One of the long standing issues I've found with persistent caching is how to juggle IO vs CPU bottlenecks. Specifically, it's faster to read one file and deserialize the JSON, than it is to do the same for multiple files. However, the larger the JSON data, the longer you'll lock the CPU up during the deserialization process.
I did some perf tests a while back to try and get an idea of the scope and constraints. I didn't get too many clear takeaways, which I suspect is mostly down to modern operating systems having amazingly fast caches.
So, from a mixture of anecdotal tests and past experience, my current plan is to use a sqlite3 db. Sqlite is portable, has some amazingly fast reads, and handles the bookkeeping side of file offsets, so that you can avoid issues surrounding the cost of deserializing stale data. Additionally, using a single sqlite db file avoids the EMFILE errors that my previous caching mechanism eventually hit.
Invalidation
I've been exploring an idea of using a file system abstraction to create a dependency graph of each compilation unit.
The current manifestation is that each call to read or stat a file is passed through a file cache that tracks the outcomes from the calls, so that when a file watcher signals a change (or a file add/remove), a set of traps can be triggered that will invalidate any units with stale dependencies.
Invalidation is also trivial during startup as you can replay the previous build's FS calls and then diff the results of isFile, mtime, and text hash checks.
While the concept does rely on some extra overhead, I'm particularly hopeful that it'll solve some long-standing complications and edge-cases that arise from caching dependency resolution.
In any case, I'm still a couple of weeks off from wrapping up the project, but I'll try to remember to do a brain dump here once I've gotten past the usual teething problems.

@mzgoddard once your plugin has stabilised, would you able to produce a little spec detailing the hooks and data structures that your plugin depends on?
My hope is that we may be able to work towards integrating your work into core or, at the least, defining some explicit APIs.
From a little dig through your source, it looked like there was a fair dependency on the implementation and structure of webpack's internals. If those hooks can be explicitly provided, it may pave the way for a robust implementation that the rest of the ecosystem can integrate with.
@amireh may also be able to shed some light based on experience with happypack.
Possibly related: https://github.com/amireh/happypack/issues/52

Gonna respond in order of your posts.

Thanks. I'm really happy that my webpack knowledge is getting used this way.

You make a really great point on storing and retrieving the cache. Reading and writing out does seem very nuanced. I took a stab at it last night with leveldb as a first shot. Mostly familiarity but figured its stream mechanism and log style writing fits the module cache's use case currently.
I am curious to try sqlite as well. Eventually it'll probably be done through plugins. I figure someone will eventually write a memcache like micro server to hold the cache in memory inbetween runs if it provides further performance gains.
The db style cache does provide some benefit (from my very anecdotal numbers). It led me to parallelize more asset and resolve cache at the same time and that definitely helped too.
I need to find an approach for assets to avoid EMFILE. I suppose that'd be just limiting the number of active ops.

Your file system abstraction approach for invalidation is really exciting. I look forward to seeing your results.

You're reading my mind. I'm definitely looking to expose hooks once I get enough tests to see what the custom version looks like that fulfill said tests. Some obvious hooks will be storing and retrieving the cache, invalidating, serializing and deserializing dependencies / assets / modules.
Whatever we can get into webpack would be great. At this point one API I'd like to see is a way to get the devtool object webpack builds. webpack builds an object that is consumed by all of the Source's source map methods (map, node, listMap, sourceAndMap) but there isn't a clear public location to get this object currently. The plugin could monkey patch those methods on the original NormalModules to hijack the info but I'd rather copy the devtool object creation than do that. Having that object from a public compiler member would also help source map support in loaders reducing source map work done to what will be consumed by webpack.
I'm not yet sure if it needs new access point in webpack 2 but I need to revisit HardSource's harmony module support. webpack 2's tree shaking leads to rendering changes in depended on modules. To support that HardSource needs a way to invalidate a module before Compilation's seal and instruct it to factory create a new Module so that the HardSource provided CacheModule is replaced by the normal build process. I haven't dug into this problem yet to confirm if this would need a new hook or not.

I have done some tiny reading on happypack. My high level understanding is it parallelizes and caches work done by loaders instead of the modules. Bet there are ways for our two plugins to work together for mutual benefit. Anything @amireh can add would be awesome.

@mzgoddard This is so so so beautiful. Your solution is so synergetic with HappyPack that I think I may have just hit happiness Rank 5.
TL;DR - HappyPack accelerates the initial build while hard-source takes over the initial build and caches it. Successive builds don't even need HappyPack or its cache anymore and are done in, as you put it, an order of magnitude shorter time.
Doing a quick test on our application at work took the (re)build time down from 90s to 29s, then 29s to 4.5s. It's just beautiful.
I don't really know how much I can add to this conversation though... HappyPack is really not as elaborate as hard-source or anything. The latter solves a much harder solution and in my opinion is a great fit to be in webpack core.
Anyway, I'll try to add my bits into what HappyPack tries to solve, how it works, and its limitations (which IMHO forbid it from being integrated into webpack core.)

Invalidation occurs at two levels: all data is namespaced during the  process's boot and the pipeline contributes a unique cache key for each file.

HappyPack adopts a somewhat similar approach but it doesn't even try to be smart about what you dub as namespacing; instead it exposes a configuration hook for defining the cache's "context" (namespace, if you will) which has to be a JSON-serializable object and delegates the responsibility to the user to tell it when to drop its cache.
One could, for example, use env-hash to populate this item.
At the file layer, it uses the file path as well as the mtime but no hashing of content - although that's also possible through another configuration hook.

Only cache the most expensive data.

This is what I realized after mucking around with webpack internals trying to generate a (de)hydratable version of the compilation state; the bulk of the time seemed to be spent on applying loader transformations.
I'll talk more about that below, which is really what HappyPack tries to improve.
On applying loaders...
What I noticed was that webpack was being CPU bound during that phase, mostly within babel transformations and node's event loop wasn't helping out much there. So what HappyPack ended up doing was spawning several processes that try to apply as many transformations as possible "in parallel" (by letting the OS figure things out) and send the results back upstream to webpack's compiler.
This process had to be transparent obviously, and the difficulty in doing that was getting loaders to be applied in a different execution environment than the one webpack compiler is running in.
The mechanism by which HappyPack tries to solve that is by providing thin proxy layers around the LoaderContext and Compiler interfaces to the loaders; these proxy layers accept requests from loaders running in background processes and forward them to the respective instances running in the foreground (owned by webpack) and then pass the results back.
The limitations of that approach are:

no support for any synchronous API (like this.resolve())
no support for any operation that relies on webpack internal objects (like
defining plugins during a loader application, which some loaders like
TypeScript's need to do in order to provide program-level reporting...)

What I wish webpack did (or didn't) do
I really wished webpack didn't choose to expose those this._compiler and this._compilation objects to loaders as they are impossible to share in a distributed environment, and sadly some loader implementations (namely, TypeScript's) mainly revolve around their utilization (even though they're documented as "hacky access") which put HappyPack in a tight corner.
I had already mentioned this to @jhnns in https://github.com/amireh/happypack/issues/52 and it was one of the reasons I didn't consider presenting something like HappyPack to webpack's core.
The other thing is it would've been nice if webpack could modularize the routines around application of loaders so that a plugin like HappyPack can reuse the implementation instead of rewriting it (this is a heavy maintenance burden really). See https://github.com/amireh/happypack/blob/master/lib/applyLoaders.js for an implementation that can be reused by both webpack and HappyPack.
What I would like to do with this...
Is take hard-source's cache and store it somewhere everytime our master branch is updated, then have a git post-pull hook download that artifact (taking the target platform in mind) and reuse it to skip the initial build!
The ultimate experience to me would be to relieve other developers of the  initial build altogether unless they're working with the build itself.
Detailed benchmarks
The following is for a single bundle with 2279 modules.

Elapsed (ms)
Happy?
Hard-Sourced?
DLLs?

95451*
No
No
No

4155

Yes

32578
Yes
No

4354

Yes

90216
No
No
Yes

29077
Yes

4511

Yes

For this project I'll be substituting happypack's cache entirely with hard-source as it's no longer necessary.

I find this figure to be strange because normally I get much lower times (~60s without happypack and 10-11s with happypack), there may be something up with my box at the moment.

@mzgoddard

I took a stab at it last night with leveldb as a first shot. Mostly familiarity but figured its stream mechanism and log style writing fits the module cache's use case currently.

I took a look at leveldb a while back: https://github.com/markfinger/unfort/issues/116#issuecomment-227311525. sqlite + leveldb seem mostly on par. I faintly remember that leveldb has (or had) patchy support for windows.

I need to find an approach for assets to avoid EMFILE. I suppose that'd be just limiting the number of active ops.

graceful-fs might be of use. It introduces a small overhead, but it'll handle the EMFILE issues.

@mzgoddard and @amireh, thank you both for all of your input, and a massive thank you for all your efforts across your projects.

With regard to persistent caching, I suspect that we should give @mzgoddard a few weeks to deal with teething issues, but once everything's settled we can sketch out a proposal and start a dialogue with @sokra to try and implement it in core.
While it may be feasible for hard-source to remain an external project, effective and performant caching will always ends up tightly coupled to webpack's implementation details. If hard-source is left external, small semver-compatible changes in webpack could easily break it. Additionally, this level of coupling puts pressure on webpack to avoid changing undocumented internals, which is an unreasonable expectation for an already large and complicated project.

Thx for your research and discussion on this topic. I really appreciate it and I'm sure we can use some of the insights to improve webpack's core. I'll bring this up at our next weekly meeting. 👍
@amireh

no support for any synchronous API (like this.resolve())

We will deprecate all synchronous APIs in loaders. They were meant as a lean possibility to also use loaders in a node.js environment (where require() calls are synchronous), but it turns out that this leads to very complicated loaders (and is not possible always). The target: node approach is probably better, although this means that we have to compile node.js code which is a great barrier for many developers.

no support for any operation that relies on webpack internal objects (like defining plugins during a loader application, which some loaders like TypeScript's need to do in order to provide program-level reporting...)

I don't know what exactly the issue was with the typescript-loader, but we definitely need to resolve their use-case and then remove this access. If it turns out that applying loaders in threads is faster because many compilation tasks a CPU bound, we should go this way.
Personally, I'd like to see some real-world figures where most time is spent in usual projects. I haven't felt that build time was a big pain for me (all my projects finish the initial build after 10s-15s), rebuilds are always under 1s. Could someone with long build times collect these figures? That would be awesome 👍

We just integrate this hard-source-plugin into Phenomic and "docs" build time is now to 13s, compared to previous 33s.

@jhnns Check out https://github.com/amireh/happypack/issues/33#issuecomment-221265755 where one of the typescript loader authors explains their use case.. it's definitely valid and I do believe that if we were able to have webpack accommodate it in a more direct approach, everyone will benefit (and then happypack's concept being ported to webpack core becomes a valid possibility.)

We will deprecate all synchronous APIs in loaders.

This would be really great.

Personally, I'd like to see some real-world figures where most time is spent in usual projects.

I am also interested in such figures because I believe there are other areas other than the application of loaders that can be optimized (and would help out in the process if I can.)
I've searched for a node profiling tool that would help me figure out where webpack is spending most of its time during build, but I didn't find any that actually worked so I have been relying on webpack's internal profiling mode so far. Do you know of any such tool? (I mean something like the gnuprof chain for C/C++ applications)

@jhnns check out https://github.com/static-dev/spike-core/issues/156. spike generates static sites using webpack as its core bundler/compiler, and adding a persistent cache feature to webpack core would, for example, directly affect my use case: compiling a massive wordpress site into a static site that I can distribute through a CDN. With a persistent cache, I can run the initial build all day if need be, and then look forward to more normal build times throughout the rest of the development process, since 90% of the static pages i generate on the first run will never get touched again.
this thread has been my first foray into webpack core, so it totally could be too much work to integrate a somewhat tangential feature. btw, this hard-source plugin looks awesome, i'm going to try integrating it asap, and that might work well enough for me. just thought I'd throw my two cents in and say that I think this would be a really great feature for anybody working on large webpack projects, including ones that are leveraging other libraries built on top of webpack (eg, spike). thanks to everyboy here for your hard work on the issue 🎉

@jhnns Any tips for breaking down time spent in various parts of the build?
Our project is massive at 451 chunks and 7217 modules, but even building a subset of that leads to long rebuild times in dev.
2016-09-02_18:28:16.85185 chunks 12
2016-09-02_18:28:16.85227 modules 1658
2016-09-02_18:28:16.85246 Build complete in 10343ms

I think flame graphs could give us better insights on where so much time is spent. I'll try to get some from my current project and post instructions on how to get them (if you do not know it anyway ^^).
We talked about this issue at our last weekly meeting. We're planing to deprecate all parts of the loader APIs that make parallel compilations impossible (namely sync APIs and _compiler/_compilation). However, in order to deprecate these we need to provide better alternatives for loaders that had to access these internal objects (namely typescript loaders but probably some others too).
@sokra had the idea of a loader API that allows to hook into different compilation states. Thus, loaders would be more like plugins but still on a per-file basis. He wanted to create a proposal so that we can discuss it with other loader authors.
Since webpack@2 already uses a separate module to execute loaders, we could write a parallel-loader which spawns multiple processes (see our meeting notes for details). The parallel-loader would need to provide a webpack-like loader context and handle all the communication in the background. I think it's a good idea to push this into user-space instead of embedding it into webpack core. This way we can keep the core simple, and parallel compilation is probably not always desired due to the costs of spawning a new process.

The proposal for the loader API is can be found here

Since webpack@2 already uses a separate module to execute loaders, we could write a parallel-loader which spawns multiple processes (see our meeting notes for details). The parallel-loader would need to provide a webpack-like loader context and handle all the communication in the background.

Honestly, that's exactly what happypack does, although it doesn't use loader-runner since at the time I saw it it didn't actually work for a distributed application (at least in happypack's context). I can't recall the details but it's probably something easily amendable.

Honestly, that's exactly what happypack does

That's perfect. You could give us valuable feedback if the proposed loader API would make it possible to use happypack rather as a loader than a plugin.

@jhnns I'm investigating our webpack 2 build that takes around 2 minutes (.tsx project with about 800 modules).
Could you please describe how to get those flame graphs?

I think flame graphs could give us better insights on where so much time is spent. I'll try to get some from my current project and post instructions on how to get them (if you do not know it anyway ^^).

@abergs run your webpack with node's inspector (a command like the following):
node --debug-brk --inspect webpack

Then launch your browser to visit the URL that command will show you (make sure you're on node 6+, 7 preferably)
In the browser page, find the Profiles panel and click on "Start Profile", then let the debugger continue (you may need to go back to the Sources panel to do this). Once the run is complete, go to the Profiles panel and click on "Stop Profile" or such, then you'll find the collected profile in a list to the left. That profile will contain the flame graph you're after.

Thank you @amireh

Is there filesystem caching yet?

Has some progress been made to make this feature at the core of webpack ?
The Rails' Asset Pipeline do it perfectly. Is it something worth investigating ?

The new webpack-contrib/cache-loader reduces consecutive builds in our environment from 50s to 27s. But you have to ensure that one cache folder isn't used for different configs (like dev/prod).

@sod that's an improvement; how much faster do you think it would be if Webpack could load its entire previous state from disk on startup, see what's changed, and then run an incremental rebuild in the exact same way it would in watch mode?  That way the time to invoke webpack again after a single file change would be reduced to just
<time to load previous state from disk> +
<time to determine which files have changed> +
<time to run incremental rebuild for the files that changed>

In my environment incremental rebuilds take on the order of 10s, and I would hope that loading the previous state and determining which files have changed take far less time.

I agree with @jedwards1211
We have like 8000 modules in our build and most of the startup time seems to be taken by reading the files and doing the acorn parsing to determine deps. We are already using a babel cache so cache-loader would not help.
I am willing to pitch in and help but hoping somebody with more experience with the codebase/problem space could take the lead or at least give a direction to get started. I think this feature would make a lot of people very happy and help alleviate performance issues.
@sokra @mzgoddard @markfinger

We have like 8000 modules in our build
Hot damn, I wonder if that's a record! 😅

That's what you get when legacy code moves to webpack, lol. It's gotten slightly smaller now, but for the longest time we had a massive hairball chunk about 50mb in size, for which i had to write a custom plugin emulating @sokra's agressive splitting plugin to split it into slices that get loaded in parallel. Good times... 😂

I was running some tests on how hard-source-webpack-plugin could be added to the Angular CLI pipeline and the results seem promising.
It does seem fairly sensitive to Webpack internals though. Since it relies on serializing Webpack sources any change in them can possibly break the plugin.
This unfortunately means that it will necessarily lag behind latest Webpack stable its changes are integrated into hard-source-webpack-plugin. Webpack 4 also seems to be bringing in a fair number of performance improvements which will probably affect sources and thus need to be integrated into the plugin.
This makes me think that the only way a plugin such as hard-source-webpack-plugin could be stable is if it's integrated directly in Webpack, or has access to a stable API (e.g sources supporting serialization). Maybe the second option would provide plugin authors with more options.

It is very old issue and duplicate more detailed issue https://github.com/webpack/webpack/issues/6527. Please leave you opinions and how we can solve this in https://github.com/webpack/webpack/issues/6527 issue. Thanks!

authors: Globegitter MagicDuck MoOx abdelbk abergs amireh bholloway damassi evilebottnawi filipesilva gdborton jedwards1211 jhnns jpea-imvu jtangelder marcello3d markfinger mzgoddard nickdima sod sokra wkentdag zkat


text:
Webpack 3 failed on shrinkwrap because of peer invalid error

Do you want to request a feature or report a bug?
bug

What is the current behavior?
After upgrade to webpack 3, my repo failed on shrinkwrap because of peer invalid
And I have created a github branch for testing
If the current behavior is a bug, please provide the steps to reproduce.

https://github.com/iroy2000/react-redux-boilerplate/tree/upgrade-to-webpack-3
Try to run
npm i and npm shrinkwrap
What is the expected behavior?
It should not fail on shrinkwrap or the built just works without peer invalid or unmet
Please mention other relevant information such as the browser version, Node.js version, webpack version and Operating System.
OS: Mac 10.12
Node: v6.10.1
npm: 3.10.0

webpack has no peerDependencies. You should report this to the package with the invalid peer dependency.

source: gharchive/issue

repo: webpack/webpack

license: MIT

license_type: permissive

license_source: github-api


text:
Troubleshooting Docker install.

So I have been attempting to install Webrecorder in my Docker environment.
I'm running CentOS Linux release 7.3.1611 (Core) with Docker version 17.03.0ce api version 1.26.
I end up with the following containers.
webrecorder_nginx_1, webrecorder_app_1, webrecorder_shepherd_1, webrecorder_proxy_1, webrecorder_redis_1, webrecorder_data_1, and webrecorder_mailserver_1.
On my first attempt webrecorder_proxy_1 and webrecorder_data_1 fail to run. The Docker log for that container show the following error "PermissionError: [Errno 13] Permission Denied: '/certs/mitproxy-ca.pem'
I found though that if I return to the folder I grabbed from git and navigate to the proxy/ and chmod aw+r certs/ and then rebuild the containers and the container stays running.
However with the webrecorder_data_1 the logs don't reveal much.
"Python 3.5.2 (default, Jan 18 2017, 02:49:23)
[GCC 4.9.2] on linux
Type "help", "copyright", "credits" or "license" for more information.

"

When I try to load http://myhost:8089/ I get a blank white page, and nginx reports a 500 error. I'm still new to docker but any help is appreciated.
QUICK EDIT: So I had the thought that there might be other permissions issues. For the sake of troubleshooting I went ahead and did a chmod 777 on the webrecorder directory then rebuilt the containers and now webrecorder starts. Obviously this isn't a good solution but I'm hoping to narrow down what permission errors are occurring.

ThankYou  , But I met the same problem with you  OS ;ubuntu 14.04.4 LTS ...

repo: webrecorder/webrecorder

license_source: bigquery


text:
Sync web page contents with the old site

Several pages are out of date when compared with the old site at https://sites.google.com/site/webrtc/.
@louquillio did you have a good way of syncing them, or is it just manual work? If so we should at least load-balance the work to get it synced up quickly.
I recall you kept a record on what changes were synced back when the new site was being developed. Maybe you can create a list of which changes needs to be synced?

It's manual. I'll do it. Signed-in admins can see recent site activity:
https://sites.google.com/site/webrtc/system/app/pages/admin/recentsiteactivity
In the past I've used this view to identify the changes, but porting them is nevertheless manual.

If PRs 25-31 are satisfactory, this can be closed.

They look good, thanks!

(I guess maybe we should wait closing until they're submitted)

authors: hkjellander louquillio

repo: webrtc/webrtc-org


text:
use native Set instead of uniq library

remove Uniq library and convert to Set and back instead

hmmm. Failing because the order is changed.
Is the order of the trackers in a magnet link important? spec doesn't say anything about order

turned out that the uniq library was sorting the array, and the Set method was actually maintaining the order from the magnet link
this is technically a change in behavior, but doesn't seem like it should be breaking to me. Probably more intuitive if anything.

source: gharchive/pull-request

repo: webtorrent/magnet-uri


text:
🛑 DragonBoat Times is down

In 83952bb, DragonBoat Times (https://www.dragonboattimes.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: DragonBoat Times is back up in 08fcf41.

repo: webworldview/uptime


text:
k8s/openshift/nginx

https://hub.docker.com/r/klzsysy/packagist-mirror

Done by #132

repo: webysther/packagist-mirror


text:
Template compiler get panicked at specific content

Reproduce:
</>

Also,
</xxx>... prevents further parsing. This is an end tag parsing bug.

authors: LastLeaf SgLy

repo: wechat-miniprogram/glass-easel


text:
bot.stop() doesn't seem to stop the accumulation of message events

When I use await bot.stop() to stop the bot service, it really won't respond to new messages.
But if a new message event arrives during the time it stops, when the bot login again, it will process these message event at one time. This means that I will send a lot of messages to one user at one time!
Does this mean that even if I stop the bot, it can still listen to the message event?
If so, how can I use bot.logonoff() for the situation that when the bot is offline , it will directly return  without processing the received message event. Like this:
bot.on("message", message);

async function message(msg) {
  if (bot.logonoff()) {
    console.log('Bot logined')
    //continue to do sth......
  } else {
    console.log('Bot not logined')
    return
  }
}

I will look forward to your reply.

This behavior highly depends on the puppet provider implementation.
There are some use cases that require the bot to process as many messages as it can when it does not online. Think about the WeChat app on your PC/Mac: it can receive messages when it's offline/logged out, to keep your message timeline complete.
In your case, it's very easy to deal with: you can use message.date() or message.age() to check whether this message was sent in the past 60 seconds.
See:

https://github.com/wechaty/wechaty/issues/1616

Ohhhh，I know how to do. Thank you!

You are welcome!

repo: wechaty/wechaty


text:
fix: fixed missing dependence & added coverage test

I'm submitting a...
[x] Bug Fix
[ ] Feature
[ ] Other (Refactoring, Added tests, Documentation, ...)

Checklist

[x] Commit Messages follow the Conventional Commits pattern

A feature commit message is prefixed "feat:"
A bugfix commit message is prefixed "fix:"

[x] Tests for the changes have been added

Description

Added missing dependences.
Added nyc for coverage test.

Does this PR introduce a breaking change?
[ ] Yes
[x] No

Thank you very much for adding the coverage test back to our testing script!

authors: Gcaufy huan


text:
设置置顶文章博客报错无法打开: database query error

这是只设置轮播, 博客正常.

如果设置了置顶

设置后刷新首页

使用空格分隔试试

使用空格分隔试试

设置单个 置顶也要加空格么?

使用空格分隔试试

设置单个 置顶也要加空格么?

单个不用加，多个用空格分开

使用空格分隔试试

设置单个 置顶也要加空格么?

单个不用加，多个用空格分开

就是设置了单个置顶

是从 release下载的, docker运行, 威联通的nas

还有你的运行环境

数据库是sqlite吗

是从 release下载的, docker运行, 威联通的nas

还有你的运行环境

数据库是sqlite吗

是的

是从 release下载的, docker运行, 威联通的nas

还有你的运行环境

数据库是sqlite吗

是的, typecho 1.2 docker版本

此问题确实是sqlite的问题，已有相同的issue #37 ，我之后会尝试测试sqlite数据库

authors: Dev-Wiki wehaox

repo: wehaox/Typecho-Butterfly


text:
LDAP Login fails when user has multiple mail attributes

Hi Team,
I'm doing some tests with Wekan on openshift, with LDAP Auth.
Login with a user with a single "mail" attribute works. Login with a user with multiple "mail" attributes doesn't.
Log :
[INFO] Init LDAP login "pcurie"
[WARN] Lookup for unset variable: INTERNAL_LOG_LEVEL
[INFO] Init setup
[INFO] Connecting "ldap://192.168.1.44:389"
[DEBUG] connectionOptions{ url: 'ldap://192.168.1.44:389',
  timeout: 10000,
  connectTimeout: 10000,
  idleTimeout: 10000,
  reconnect: true,
  log:
   Logger {
     domain: null,
     _events: {},
     _eventsCount: 0,
     _maxListeners: undefined,
     _level: 30,
     streams: [ [Object] ],
     serializers: null,
     src: false,
     fields:
      { name: 'ldapjs',
        component: 'client',
        hostname: 'wekan-14-b6slr',
        pid: 1 } } }
[INFO] LDAP connected
[INFO] Binding UserDN "uid=svc-wekan,cn=sysaccounts,cn=etc,dc=acme,dc=org"
[INFO] Searching user "pcurie"
[DEBUG] searchOptions {
  "filter": "(&(uid=pcurie))",
  "scope": "sub",
  "sizeLimit": 0
}
[DEBUG] BaseDN "cn=users,cn=accounts,dc=acme,dc=org"
[INFO] Search result count 1
[INFO] Authenticating "uid=pcurie,cn=users,cn=accounts,dc=acme,dc=org"
[INFO] Authenticated "uid=pcurie,cn=users,cn=accounts,dc=acme,dc=org"
[DEBUG] Identifying user with: uid
[INFO] Querying user
[DEBUG] userQuery {
  "services.ldap.id": "706175637572"
}
[DEBUG] userQuery {
  "username": "pcurie"
}
[INFO] User does not exist, creating "pcurie"
[DEBUG] Identifying user with: uid
[DEBUG] Mapping field cn -> name
[DEBUG] user.name changed to: Paul CURIE
[DEBUG] Mapping field mail -> email
[DEBUG] Identifying user with: uid
[DEBUG] New user data {
  "username": "pcurie",
  "email": {
    "type": "Buffer",
    "data": [
      112,
      97,
      117,
      99,
      117,
      114,
      64,
      119,
      111,
      114,
      116,
      101,
      107,
      115,
      46,
      99,
      111,
      109
    ]
  }
}
[ERROR] Error creating user {
  "message": "Match error: Expected string, got object in field email",
  "path": "email",
  "sanitizedError": {
    "isClientSafe": true,
    "error": 400,
    "reason": "Match failed",
    "message": "Match failed [400]",
    "errorType": "Meteor.Error"
  },
  "errorType": "Match.Error"
}
Exception while invoking method 'login' Error: Match error: Expected string, got object in field email
    at exports.check (packages/check.js:55:15)
    at createUser (packages/accounts-password/password_server.js:1028:3)
    at AccountsServer.Accounts.createUser (packages/accounts-password/password_server.js:1123:10)
    at addLdapUser (packages/wekan:wekan-ldap/server/sync.js:277:31)
    at DDPCommon.MethodInvocation.<anonymous> (packages/wekan:wekan-ldap/server/loginHandler.js:159:18)
    at packages/accounts-base/accounts_server.js:468:32
    at tryLoginMethod (packages/accounts-base/accounts_server.js:245:14)
    at AccountsServer.Ap._runLoginHandlers (packages/accounts-base/accounts_server.js:465:18)
    at DDPCommon.MethodInvocation.methods.login (packages/accounts-base/accounts_server.js:528:27)
    at packages/check.js:128:16
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:15)
    at Object._failIfArgumentsAreNotAllChecked (packages/check.js:127:41)
    at maybeAuditArgumentChecks (packages/ddp-server/livedata_server.js:1765:18)
    at DDP._CurrentMethodInvocation.withValue (packages/ddp-server/livedata_server.js:719:19)
    at Meteor.EnvironmentVariable.EVp.withValue (packages/meteor.js:1186:15)
    at DDPServer._CurrentWriteFence.withValue (packages/ddp-server/livedata_server.js:717:46)
Sanitized and reported to the client as: Match failed [400]

[INFO] Idle
[INFO] Disconecting
[INFO] Closed

Thank you

As a work-around, you can remove "LDAP_EMAIL_FIELD" variable from your env, and wekan let you login.

Interesting - is that the only change you made to get it working?
I can't see how removing the environment variable fixes the issue.

Indeed, sorry I wasn't careful enough in my tests.
What actually made it work was setting "LDAP_SYNC_USER_DATA_FIELDMAP" from {"displayName":"name","mail":"email"} to {"displayName":"name"}.
So it seems related to sync fonctions.

Yes, that makes sense considering it will no longer attempt to sync email address.
Out of interest, could you expand a bit on the multiple mail attributes? Do you have multiple attributes for the user with the same name (i.e. more than one "email" attribute) or is it a multi valued attribute?
Seeing an example LDAP user entry would be helpful.

It is multi-valued attribute, in ldif format it would look like :
mail: short@domain.tld
mail: long.mail@domain.tld

authors: pcurie stevenpwaters

repo: wekan/wekan-ldap


text:
Videos issues on https://wellcomecollection.org/works/

@jtweed
Issues noted on spot checking video titles on https://wellcomecollection.org/works/

Mapping of video Sierra records to the API  https://wellcomecollection.org/works/

AV titles have always had a separate e-bib record for the digitised file.
Example 1: Cruel Kindness
A search in wellcomelibrary.org will return 3 records for this title.
i) The film reel: https://search.wellcomelibrary.org/iii/encore/record/C__Rb1656590__Scruel kindness__Orightresult__X7?lang=eng&suite=cobalt
ii) Any physical video formats (in this case DVDs and Digibetas): https://search.wellcomelibrary.org/iii/encore/record/C__Rb1658712__Scruel kindness__P0%2C1__Orightresult__X7?lang=eng&suite=cobalt
iii) E-bib record for digitised item: https://search.wellcomelibrary.org/iii/encore/record/C__Rb1667616__Scruel kindness__P0%2C2__Orightresult__X7?lang=eng&suite=cobalt
We use the e-bib record to capture the license information in the 540 field for the digitised title (in this case CC BY NC)
However, on https://wellcomecollection.org/works, the search returns 2 records:
i) The film reel.  This is as it should be.
ii) A result for video. This shows the digitised file, but associated record has been pulled from the Sierra record for the physical format: https://wellcomecollection.org/works/mgsp56ab
The anomaly was first noticed when I changed the license in the Sierra e-bib for another title Sir Henry Dale and couldn’t see it update on Works.

‘View the item’ option
The ‘view the item’ button currently appears with all videos on Works, even if we are not planning to make the title available online, e.g. Nystagmus in epilepsy https://wellcomecollection.org/works/ckkcfmt9
The above title has been digitised but is a restricted item due to data protection of clinical content: https://wellcomelibrary.org/item/b30467883#?c=0&m=0&s=0&cv=0  The e-bib record is suppressed and hence does not appear on wellcomelibrary.org
The Works API  has pulled the record for the physical video format: https://wellcomecollection.org/works/ckkcfmt9 This allows you to click on the ‘View the item’ button, which takes you to the message’ 'We are working to make this item available online. But, in fact, we won’t be making this item available online.

A similar issue arises for videos which have not yet been digitised, e.g. The Russian Arm https://wellcomecollection.org/works/w72fw4n5 Again, there is a ‘View the item’ button linked to the record of the physical video asset, but it may be years before a title is digitised.

Hi @HarkiranDhindsa. We're going to fix this in the short term by not merging AV works. That's covered by #4876, so I'm going to close this ticket and we can track there.
Longer term, we'll work with Collection Information on how they intend to catalogue AV going forward, which will have to change not least because some of the high-res digitisation material will become the master format.
When we know that, we'll work out how best to merge or otherwise group AV. In the meantime, there will be a work for each format like there is in Encore.

This is now fixed; searching for "Cruel kindness" on /works turns up three records, which is the desired behaviour AIUI: https://wellcomecollection.org/works?page=1&query=Cruel Kindness

This is now fixed; searching for "Cruel kindness" on /works turns up three records, which is the desired behaviour AIUI: https://wellcomecollection.org/works?page=1&query=Cruel Kindness

Thanks, good to see the the separate records.
But the video records seems to be linked the wrong way round. b16676166 should be the e-bib for the digitised item with the online link. b16587121 should be the catalogue record for the physical video formats.
For comparison, this is how they appear on Wellcome Library: https://search.wellcomelibrary.org/iii/encore/search/C__Scruel kindness__Orightresult__U?lang=eng  The e-bib is much more detailed with info for the segments of film.

Thanks, good to see the the separate records.
But the video records seems to be linked the wrong way round. b16676166 should be the e-bib for the digitised item with the online link. b16587121 should be the catalogue record for the physical video formats.
For comparison, this is how they appear on Wellcome Library: https://search.wellcomelibrary.org/iii/encore/search/C__Scruel kindness__Orightresult__U?lang=eng  The e-bib is much more detailed with info for the segments of film.

I'm not quite sure how we could have got the two flipped round; reopening until we can investigate further.

I'm not quite sure how we could have got the two flipped round; reopening until we can investigate further.

I put in some fixes for this on Friday, and Harkiran has confirmed in Slack that this seems to be okay. I also can't see the unwanted "View item" button any more, so closing.

authors: HarkiranDhindsa alexwlchan jtweed

repo: wellcomecollection/platform


text:
Update to the newest version of scalatest

This is required to get the latest elastic4s, which we need for https://github.com/mockito/mockito/issues/945

This is done.


text:
Add a toggle for iiif-test.wc.org

As part of the work on the new-and-improved DLCS, Tom Crane has spun up a new instance at iiid-test.wellcomecollection.org, e.g. https://iiif-test.wellcomecollection.org/presentation/b16641097
To check the site works correctly with this new version of DLCS and the slightly modified IIIF Presentation manifests, we should create a toggle that causes the site to use the new URLs.

Created!


text:
Fix the Sierra lettering transformation

As discussed in this morning's domain modelling session:
Currently we look for 216 ind 16, then populate from fixed field "lettering". Two problems:

We don't care about the value of the 1st indicator
There is no "lettering" fixed field!

We should look for 216, 2nd ind 6, then populate from subfield $a.

Small correction 246 I think in this case '

authors: alexwlchan silveroliver

repo: wellcometrust/platform


text:
The Miro adapter is pining for the fjords

What is this PR trying to achieve?
Now we have the Miro preprocessor pipeline, this adapter unnecessary. It is no more. It has ceased to be. It is an ex-adapter. The only reason it’s still in the repo is because it’s been nailed there.
So, let’s delete it instead.
Who is this change for?
A platform team who want a neat and tidy repo.
Have the following been considered/are they needed?

[ ] Deployed new versions
[ ] Run terraform apply.

👍 Will wait for proof we can reproduce this behaviour with the new pipeline before merging.

I reckon we’re close enough and I want to get this in before the other build changes, merging.

authors: alexwlchan kenoir


text:
kubevpn dev check resource before running

I got docker 137 return code.
Can kubevpn check if docker desktop resource config satisfy remote k8s resource limit, and log WARN while not?

good idea, i will add precheck logic next version

already add precheck logic, commit: https://github.com/wencaiwulue/kubevpn/commit/30a82e25eb7e2792f9f76716fb161d9c078b98f3

authors: ealyn wencaiwulue

repo: wencaiwulue/kubevpn


text:
decoder_main 支持 dfsmn的解码吗？

你好，decoder_main 是否支持 dfsmn结构的模型解码？

only suppirt ctc 、  wfst、 attention reacoring

您好，您的邮件已经收到，尽快给您回复。

authors: Mddct xiaosimao


text:
运行runtime/gpu的readme案例时，出现unexpected keyword argument 'model_name'问题

一、Bug描述

使用wenet/tree/main/runtime/gpu进行readme的案例运行时，server开启正常，client开启容器正常，但运行client案例时（python3 client.py --audio_file=/ws/test_data/mid.wav --url=0.0.0.0:8001），发生以下问题（unexpected keyword argument 'model_name'）：
client提示：

server提示：

初步定位到（/ws/model_repo/scoring/1/model.py(246)）：

二、环境验证
2.1、server端

server端开启正常：

server端环境配置：

server端容器环境:

nvcr.io/nvidia/tritonserver:21.04-py3：
cuda环境：

server端容器python环境:

2.2、client端

镜像：nvcr.io/nvidia/tritonserver:21.04-py3-sdk

请问是什么原因使用21.04呢？没有这个Model_name可能是那个时候的triton确实没有，试试 22.12 或者更新的23.0x吧

请问是什么原因使用21.04呢？没有这个Model_name可能是那个时候的triton确实没有，试试 22.12 或者更新的23.0x吧

使用21.04主要是对应cuda11.3；
线下已与@yuekaizhang大佬沟通，如果需要使用cuda11.3，建议拉取大佬个人打包的镜像docker pull soar97/triton-wenet:22.12；

补充一下：tritonserver 官方镜像 22.xx系列都是支持 cuda 11.3的， soar97/triton-wenet:22.12 是基于 tritonserver 22.12编译的

authors: lvchigo yuekaizhang


text:
inference Error

When I call inference, I got  an error:
Traceback (most recent call last):
File "/home/forever/PycharmProjects/PIG/resnet.py", line 313, in
conv9_out = inference(x, FLAGS.num_residual_blocks, reuse=False)
File "/home/forever/PycharmProjects/PIG/resnet.py", line 194, in inference
assert conv3.get_shape().as_list()[1:] == [8, 8, 64]
AssertionError
can you tell where should I change my code?

did u solve this error? I got this error too.

authors: HandsomeHans chunguGuo

repo: wenxinxu/resnet-in-tensorflow


text:
第三方权限、回调问题，比如极光、微信

1、第三方权限问题。插件中的权限需要在宿主的清单中声明，但是有些第三方平台申请的权限是自定义权限，要带上包名，这里的包名是宿主的还是插件的。
2、第三方的回调。还有就是第三的一些回调应该在宿主中，还是插件里面。比如微信支付成功的回掉，极光接收到消息的回调。

1.“自定义权限”这说的是什么？
2.按照Small使用中间产物的原理来看，合并好的manifest中间产物里已经包含了第三方模块的manifest所注册的内容，所以应该不存在你说的这个问题，试试看

第三方SDK插件的权限等的 需要配置在宿主menifest中

@WooYu  你的极光Manifest配置是放在哪里？放在Lib里，可行吗？

@WooYu @kuwork ，极光推送集成成功了吗？

@dianyueneo 可以了

@kuwork  請問下，你是怎麽解決的微信回調這些。

@AndroidDevelopersLiqingwen 应该和极光一样在，宿主配置。

@kuwork 请问集成极光推送的 compile 依赖的库放在宿主里吗，还是自定义的Receiver要放在哪里？

authors: AndroidDevelopersLiqingwen WooYu dianyueneo eric0614 kuwork offbye xiaweizi

repo: wequick/Small


text:
Quality of life change- Added cursor:Pointer when hovering on table b…

added cursor: 'pointer' in styles.css for Quality changes

Thanks for your contribution!

authors: palashmon revengemiroz

repo: wesbos/keycodes


text:
Internationalisation (I18N)

Currently the settings screen of a user allows to switch between English and Japanese.
When selecting English, still some Japanese artifacts remain in the user interface.
It would be nice to have internationalisation configurable from the admin settings, where one could choose standard languages for all users, choose if language switching is available. Also preparing a Transifex/Weblate project can help in translating user interface strings into other languages.
I am taking the effort to write this issue, since Growi is the most feature-complete Markdown Wiki that I found during the last years searching for one, and I wish it receives wide adoption.
Internationalisation will help in allowing users and developers outside of Japan to interact more easily.

@almereyda Thank you for your post.

I am taking the effort to write this issue, since Growi is the most feature-complete Markdown Wiki that I found during the last years searching for one, and I wish it receives wide adoption.

We are really encouredged your message.
I agree with the importance of internationalization.
Since I recognized that GROWI users are only Japanese people, I have not paid attention before.

Also preparing a Transifex/Weblate project can help in translating user interface strings into other languages.

I didn't know about Transifex/Weblate. We consider it. Thanks.

For convenience, you can find both tools at

https://www.transifex.com/ (commercial)
https://weblate.org/ (FLOSS)

If you wish to try out the latter, we're running an instance at https://weblate.allmende.io/

It looks like Transifex provides a free license for open source projects.
I inquired about it.

I created Transifex Project for the time being.
https://www.transifex.com/weseek-inc/growi

authors: almereyda yuki-takei

repo: weseek/growi


text:
The records factory should choose the correct header automatically

Currently there are two methods for creating records, one for transaction records, another for non transaction records.
The factory should automatically build one or the other header. This can be set on the record configuration file.

Done.

repo: weso/CWR-DataApi

license: mit


text:
Update scala3-library to 3.1.2

Updates org.scala-lang:scala3-library from 3.1.1 to 3.1.2.
GitHub Release Notes - Version Diff - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (3.1.1).
You might want to review and update them manually.
src/test/resources/shacl/tests/core/complex/shacl-shacl-data-shapes.ttl

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "org.scala-lang", artifactId = "scala3-library" } ]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

Superseded by #518.

repo: weso/shaclex


text:
run scenarios with a tag

Add a new flag named tags  to the command run.  This flag will be used to determine which scenarios must be executed.
For example, given the below input file
scenario "basic usage" {
  tags= ["sum","maths"]
  given "a couple of numbers" {
    set a {
      value = 1
    }
    set b {
      value = 2
    }
  }
  when "input values are sum" {
    set c {
      value = a + b
    }
  }
  then "the result of variable c is correct" {
    assert {
      assertion = c == 3
    }
  }
}

scenario "basic usage" {
  tags= ["sub", "maths"]
  given "a couple of numbers" {
    set a {
      value = 1
    }
    set b {
      value = 2
    }
  }
  when "do subtraction with the numbers" {
    set c {
      value = a - b
    }
  }
  then "the result of variable c is correct" {
    assert {
      assertion = c == -1
    }
  }
}

the below command should work, and only scenarios with provided tags should be executed.
orion  run --input feature.hcl --tags sum
orion  run --input feature.hcl --tags sub
orion  run --input feature.hcl --tags maths
orion  run --input feature.hcl --tags sum,sub

Done by @cdmatta

repo: wesovilabs/orion


text:
[WV-337] Convert Form Layout Changes to Include Templates (office held) TEAM REVIEW

Updated so dict values are passed into the form for new offices held (before, labels weren't showing on that form)
Removed the template values that were no longer needed now that dicts were used for them

👍

repo: wevote/WeVoteServer


text:
Add Cross.toml in order to pass the NIF version to cross

This is important because cross don't have access by default to the
env vars in the system.
This is related to https://github.com/philss/rustler_precompiled/issues/23

Oh nice! 👀

authors: philss whatyouhide


text:
Storeのスクリーンショット作成

対応内容
一旦必要なスマホ画像の素材を作成して共有する

https://mockuphone.com/device#iphonese

repo: wheatandcat/memoir


text:
Fix(popover.js): destroy check for trigger & classes

This is copy of this PR.
TODO:

[ ] Run the build scripts.

Oh sorry, this is the wrong repo :(.

repo: wheldring/bootstrap-vue-compiled


text:
Fetch is undefined after updating to version 10.0.6 in jest 29.7.0

Hi everyone! 👋
I'm experiencing an issue with fetch-mock while using Jest version 29.7.0.
Problem Description:
After updating fetch-mock from version 9.11.0 to 10.0.6, I'm encountering the following problem:

It appears that the issue is due to a lack of native support for fetch. I've tried a few solutions, such as:
jest.mock('node-fetch', () => require('fetch-mock').sandbox())

global.fetch = jest.fn(() =>
  Promise.resolve({
    json: () => Promise.resolve({ test: 100 }),
  }),
) as jest.Mock;

Unfortunately, these solutions didn't work. Adding the whatwg-fetch package did solve the problem, but I don't think it's ideal to add another dependency just for this.
🔍 Can anyone help me with a better solution?
Thanks in advance! 🙏

I get this error as well.  When I have testEnvironment: "jsdom" in jest.config.ts, fetchMock.config.Request is undefined:
console.log(fetchMock);

fetchMock {
      _uncompiledRoutes: [],
      routes: [],
      fallbackResponse: undefined,
      config: {
        fallbackToNetwork: false,
        includeContentLength: true,
        sendAsJson: true,
        warnOnFallback: true,
        overwriteRoutes: undefined,
        Request: undefined,
        Response: undefined,
        Headers: [class Headers],
        fetch: undefined
      },
      _calls: [],
      _holdingPromises: [],
      fetchHandler: [Function: bound ],
      restore: [Function: bound ],
      reset: [Function: bound ],
      resetHistory: [Function: bound ],
      resetBehavior: [Function: bound ]
    }

If I remove testEnvironment: "jsdom" then the test runs fine:
console.log(fetchMock);

fetchMock {
      _uncompiledRoutes: [],
      routes: [],
      fallbackResponse: undefined,
      config: {
        fallbackToNetwork: false,
        includeContentLength: true,
        sendAsJson: true,
        warnOnFallback: true,
        overwriteRoutes: undefined,
        Request: [class Request],
        Response: [class Response] {
          error: [Function: error],
          json: [Function: json],
          redirect: [Function: redirect]
        },
        Headers: [class Headers],
        fetch: [Function: fetch]
      },
      _calls: [],
      _holdingPromises: [],
      fetchHandler: [Function: bound ],
      restore: [Function: bound ],
      reset: [Function: bound ],
      resetHistory: [Function: bound ],
      resetBehavior: [Function: bound ]
    }

@gabrieldeespindula Unless something is forcing you on to v10 I would just stick with v9 for now. I'm currently working on a big rewrite to ease compatibility problems more broadly, and don't want to spend time on compatibilty issues with the existing versions if I can avoid it.

repo: wheresrhys/fetch-mock


text:
Clan Support

Hey really enjoyed the plugin!
I went and implemented the new chats menu entries and highlighting.
I also added when someone is invited to the clan and accepts it auto checks if they are in the list.

Awesome! I'll merge this in tomorrow!

repo: while-loop/runelite-plugins


text:
Support HTML embed

This can work using oEmbed as well as pasting HTML
The embed would have a default size that can be overridden either by the embed code or by manually inputing a new field
For this content type will also be able to include a custom SVG icon

Reference: https://iframely.com/try/

repo: whiletrue-industries/Chronomaps


text:
Run client validation on save

@rpominov

👍

authors: faergeek rpominov

repo: whitescape/react-admin-components


text:
Inheritance fixes

Hey Whoahbot!
I found an earlier approach by ujifgc for supporting inheritance. I did a cherry pick from his branch and it seems to work well. Maybe you  could give it a try.
Cheers

Please, keep in mind that https://github.com/sfeu/dm-redis-adapter/commit/cb0173ce588061686d71156e06a1c3de079035f4 changes every storage name from singular to plural form. This should be addressed in the readme or changelog.

authors: sfeu ujifgc

repo: whoahbot/dm-redis-adapter


text:
参考文献中作者人名含重音

参考文献中作者名含有重音导致出现下面的情况

请问这样符合标准吗？

7714 中似乎没有出现过带重音符号的外文人名，但我猜这里的字母也需要转为大写（这一点需要与 @zepinglee 老师确认）
测试发现，如果使用 bibtex，只有使用 \"o 之类的转义符号（而不是直接使用 ö），可以自动转为大写；同时，如果使用 biblatex，则无论使用转义还是直接使用 ö，都能自动转为大写

7714 中似乎没有出现过带重音符号的外文人名，但我猜这里的字母也需要转为大写（这一点需要与 @zepinglee 老师确认）
测试发现，如果使用 bibtex，只有使用 \"o 之类的转义符号（而不是直接使用 ö）才可以自动转为大写；同时，如果使用 biblatex，则无论使用转义还是直接使用 ö，都能自动转为大写

直接使用转义符号确实变为大写，感谢。

BibTeX 不支持 unicode，所以只有用 \"o 才能转换大小写。

明白了，谢谢解答！

authors: SwitWu tanukihee zepinglee

repo: whutug/whu-thesis


text:
Failed to run "single_video_inference.py"

Thanks for your interesting work~
I try to run single_video_inference.py but the program returns nothing. Besides, I noticed the  D2 aggregation codes in "def temporal_aggregation()" have been canceled. So could you give some guidance to run your program?  Thanks.

Thank you for your interest in my work!
The code defaults to using simplest D1. Please change the "num_frames" from 8 to 4.
If you need to use D2 aggregation, comment D1 and uncomment D2. Then, the "num_frames" can be increased to 8.


text:
增加一个 解压缩文件功能

Clear and concise description of the problem
有大量文件需要上传的 时候 只能一个个上传 希望增加一个批量上传 或者解压缩功能
Suggested solution
无
Alternative
No response
Additional context
No response
Validations

[X] Check that there isn't already an issue that request the same feature to avoid creating a duplicate.

可以放到一个git仓库里去克隆

authors: okeyzero whyour

repo: whyour/qinglong


text:
🛑 Le Coq 🇬🇧 (en) is down

In 400bed3, Le Coq 🇬🇧 (en) (https://holiday-lescala.com/en/h/house-le-coq/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Le Coq 🇬🇧 (en) is back up in 6df484b after 10 minutes.

repo: whytspace/upptime-holiday-lescala


text:
🛑 Übersicht 🇬🇧 (en) is down

In a75b11d, Übersicht 🇬🇧 (en) (https://holiday-lescala.com/en/homes/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Übersicht 🇬🇧 (en) is back up in 6ce61a6 after 37 minutes.


text:
Gestión de errores

gestionar los inputs inválidos, etc etc

Se acaba esta semana

repo: wiinterrr/Editor-de-fotos


text:
More user-friendly error message by parsing response

Bug: T138255

I've got one note, but I'm OK with merging this as is since it is an improvement.

authors: Ladsgroup halfak

repo: wiki-ai/wikilabels


text:
minor - getting some ansi on eclipse console

Ah. The default is color formatting. We can probably be smarter about TTY handling maybe.
If you configure the LoggingFraction you can setup a simpler formatted without color.
--
Sent from my iPhone

On Jun 18, 2015, at 9:55 PM, Helio Frota notifications@github.com wrote:
—
Reply to this email directly or view it on GitHub.

@eprogramming
Hmm, It's strange. In wf-swarm default logging setting, its named-formatter attr is PATTERN, not "COLOR-PATTERN".
Did you change it ?
Related entry
Disabling color escape sequences in WildFly's console logging for IDE

If you configure the LoggingFraction you can setup a simpler formatted without color.

going to learn how to do it.

Hmm, It's strange. In wf-swarm default logging setting, its named-formatter attr is PATTERN, not "COLOR-PATTERN".

got it, yup is a bit weird.

Did you change it ?

No
main class example

Ah, I'm with you. The color formatted log comes from boot logger whose formatter is "COLOR-PATTERN".
See WildFly Distribution logging.properties($WILDFLY_HOME/standalone/configuration/logging.properties).
If you change it, you need to overwrite it, maybe. Or, are there any way to do it?

@emag i got it, thanks
this right ? handler.CONSOLE.formatter=COLOR-PATTERN
and the blog entry was using 'named-formatter' maybe this is a 'custom handler attribute' ?
[wildscribe] (http://wildscribe.github.io/Wildfly/9.0.0.Beta1/subsystem/logging/custom-handler/index.html)
btw was good to learn how to configure LoggingFraction and ofc enable debug mode
container.subsystem(new LoggingFraction()
.formatter("PATTERN", "%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n")
.consoleHandler("DEBUG", "PATTERN")
.rootLogger("CONSOLE", "DEBUG"));

logs

@eprogramming

this right ? handler.CONSOLE.formatter=COLOR-PATTERN

Yes, exactly.

and the blog entry was using 'named-formatter' maybe this is a 'custom handler attribute' ?

No, it's console-logger(and, it's runtime logger) attr. wildfly-swarm settings is 'PATTERN' while distro is 'COLOR-PATTERN' such like,
[standalone@localhost:9990 /] /subsystem=logging/console-handler=CONSOLE:read-attribute(name=named-formatter)
{
    "outcome" => "success",
    "result" => "COLOR-PATTERN"
}

the blog guy uses WildFly distro.

btw was good to learn how to configure LoggingFraction and ofc enable debug mode

Nice :)
And you can use 'COLOR-PATTERN'. Of course, unfortunately, it's not Eclipse freindly...
...
.consoleHandler("DEBUG", "COLOR-PATTERN")
...

@eprogramming
I found a good plugin. I think you'd like it.
https://github.com/mihnita/ansi-econsole
My LoggignFraction setting is ...
container.subsystem(new LoggingFraction()
	.formatter("PATTERN", "%d{yyyy-MM-dd HH:mm:ss,SSS} %-5p [%c] (%t) %s%e%n")
	.consoleHandler(DEFAULT_LEVEL, "COLOR-PATTERN")
	.rootLogger("CONSOLE", DEFAULT_LEVEL));

@emag yup i know this plugin before is good btw.
about the ansi escape code :
\u001B[0m

this means is only trying to reset
so that, this is a issue from eclipse console not wildfly-swarm, and i guess this issue here can be closed : ]

We've moved to the name COLOR_PATTERN and PATTERN for the color/ansi formatter and non-color, respectively.
It defaults to using COLOR_PATTERN, but we've also made it easier to change.

authors: bobmcwhirter emag eprogramming

repo: wildfly-swarm/wildfly-swarm


text:
[WFCORE-5349] Upgrade to WildFly Elytron 1.15.2.Final

https://issues.redhat.com/browse/WFCORE-5349
https://github.com/wildfly-security/wildfly-elytron/compare/1.15.1.Final...1.15.2.Final
    Release Notes - WildFly Elytron - Version 1.15.2.Final

        Bug

[ELY-2104] -         two calls to introspection endpoint for one request with JWT

        Release

[ELY-2106] -         Release WildFly Elytron 1.15.2.Final

@darranl thanks!

authors: darranl jmesnil

repo: wildfly/wildfly-core


text:
[WFCORE-5580] Fix conditional structure (controller)

Fixes https://issues.redhat.com/browse/WFCORE-5580

Core - Full Integration Build 10954 outcome was FAILURE using a merge of 62da6381bbbd5c5305e36bc25a03834f265d8b4e
Summary: Tests failed: 1 (1 new), passed: 7276, ignored: 132 Build time: 03:09:35
Failed tests
org.jboss.as.test.compat.jpa.hibernate.HibernateJarsInDeploymentTestCase:     Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR [\"/content/HibernateJarsInDeploymentTestCase.ear\"]
    Caused by: java.io.IOException: No space left on device"}}}}
Cannot deploy HibernateJarsInDeploymentTestCase.ear: {"WFLYCTL0062: Composite operation failed and was rolled back. Steps that failed:" => {"Operation step-1" => {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"HibernateJarsInDeploymentTestCase.ear\".STRUCTURE" => "WFLYSRV0153: Failed to process phase STRUCTURE of deployment \"HibernateJarsInDeploymentTestCase.ear\"
    Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR [\"/content/HibernateJarsInDeploymentTestCase.ear\"]
    Caused by: java.io.IOException: No space left on device"}}}}
	at org.jboss.as.arquillian.container.ArchiveDeployer.deployInternal(ArchiveDeployer.java:184)
	at org.jboss.as.arquillian.container.ArchiveDeployer.deployInternal(ArchiveDeployer.java:162)
	at org.jboss.as.arquillian.container.ArchiveDeployer.deploy(ArchiveDeployer.java:91)
	at org.jboss.as.arquillian.container.CommonDeployableContainer.deploy(CommonDeployableContainer.java:255)
------- Stdout: -------
 [0m02:16:24,008 INFO  [org.jboss.as.repository] (management-handler-thread - 2) WFLYDR0001: Content added at location /store/work/tc-work/37b47ae8b9c60325/full/testsuite/compat/target/wildfly/standalone/data/content/5a/dfd5ca4935daa4bcd404cc39e3057b2b026a4f/content
 [0m [0m02:16:24,011 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-4) WFLYSRV0027: Starting deployment of "HibernateJarsInDeploymentTestCase.ear" (runtime-name: "HibernateJarsInDeploymentTestCase.ear")
 [0m [31m02:16:24,045 ERROR [org.jboss.msc.service.fail] (MSC service thread 1-3) MSC000001: Failed to start service jboss.deployment.unit."HibernateJarsInDeploymentTestCase.ear".STRUCTURE: org.jboss.msc.service.StartException in service jboss.deployment.unit."HibernateJarsInDeploymentTestCase.ear".STRUCTURE: WFLYSRV0153: Failed to process phase STRUCTURE of deployment "HibernateJarsInDeploymentTestCase.ear"
	at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:189)
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.startService(ServiceControllerImpl.java:1739)
	at org.jboss.msc.service.ServiceControllerImpl$StartTask.execute(ServiceControllerImpl.java:1701)
	at org.jboss.msc.service.ServiceControllerImpl$ControllerTask.run(ServiceControllerImpl.java:1559)
	at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35)
	at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1990)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486)
	at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377)
	at java.lang.Thread.run(Thread.java:748)
Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR ["/content/HibernateJarsInDeploymentTestCase.ear"]
	at org.jboss.as.ee.structure.EarStructureProcessor.deploy(EarStructureProcessor.java:247)
	at org.jboss.as.server.deployment.DeploymentUnitPhaseService.start(DeploymentUnitPhaseService.java:182)
	... 8 more
Caused by: java.io.IOException: No space left on device
	at java.io.FileOutputStream.writeBytes(Native Method)
	at java.io.FileOutputStream.write(FileOutputStream.java:326)
	at org.jboss.vfs.VFSUtils.copyStream(VFSUtils.java:439)
	at org.jboss.vfs.VFSUtils.copyStream(VFSUtils.java:418)
	at org.jboss.vfs.TempDir.createFile(TempDir.java:86)
	at org.jboss.vfs.spi.JavaZipFileSystem.<init>(JavaZipFileSystem.java:77)
	at org.jboss.vfs.VFS.mountZip(VFS.java:386)
	at org.jboss.vfs.VFS.mountZip(VFS.java:410)
	at org.jboss.as.ee.structure.EarStructureProcessor.mount(EarStructureProcessor.java:253)
	at org.jboss.as.ee.structure.EarStructureProcessor.deploy(EarStructureProcessor.java:132)
	... 9 more

 [0m [31m02:16:24,051 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 2) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "HibernateJarsInDeploymentTestCase.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"HibernateJarsInDeploymentTestCase.ear\".STRUCTURE" => "WFLYSRV0153: Failed to process phase STRUCTURE of deployment \"HibernateJarsInDeploymentTestCase.ear\"
    Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR [\"/content/HibernateJarsInDeploymentTestCase.ear\"]
    Caused by: java.io.IOException: No space left on device"}}
 [0m [31m02:16:24,051 ERROR [org.jboss.as.controller.management-operation] (management-handler-thread - 2) WFLYCTL0013: Operation ("add") failed - address: ([("deployment" => "HibernateJarsInDeploymentTestCase.ear")]) - failure description: {"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"HibernateJarsInDeploymentTestCase.ear\".STRUCTURE" => "WFLYSRV0153: Failed to process phase STRUCTURE of deployment \"HibernateJarsInDeploymentTestCase.ear\"
    Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR [\"/content/HibernateJarsInDeploymentTestCase.ear\"]
    Caused by: java.io.IOException: No space left on device"}}
 [0m [31m02:16:24,052 ERROR [org.jboss.as.server] (management-handler-thread - 2) WFLYSRV0021: Deploy of deployment "HibernateJarsInDeploymentTestCase.ear" was rolled back with the following failure message:
{"WFLYCTL0080: Failed services" => {"jboss.deployment.unit.\"HibernateJarsInDeploymentTestCase.ear\".STRUCTURE" => "WFLYSRV0153: Failed to process phase STRUCTURE of deployment \"HibernateJarsInDeploymentTestCase.ear\"
    Caused by: org.jboss.as.server.deployment.DeploymentUnitProcessingException: WFLYEE0054: Failed to process children for EAR [\"/content/HibernateJarsInDeploymentTestCase.ear\"]
    Caused by: java.io.IOException: No space left on device"}}
 [0m [0m02:16:24,053 INFO  [org.jboss.as.server.deployment] (MSC service thread 1-1) WFLYSRV0028: Stopped deployment HibernateJarsInDeploymentTestCase.ear (runtime-name: HibernateJarsInDeploymentTestCase.ear) in 0ms
 [0m

@yersan thanks.
We can merge this one: The Full intregation -Linux job is passing and I don't see changes in that PR that would be Windows-specific.

authors: boris-unckel jmesnil wildfly-ci


text:
WFLY-3991 javax.script.ScriptEngineFactory: rhino provider not found

since we minimum requirement is now JDK8 no need to include class name for Rhino

Windows Build 1447 is now running using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643

Linux Build 1895 is now running using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643

Core - Full Integration Build 1036 is now running using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643

Windows Build 1447 outcome was SUCCESS using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643
Summary: Tests passed: 3122, ignored: 71 Build time: 0:20:25

Linux Build 1895 outcome was SUCCESS using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643
Summary: Tests passed: 3122, ignored: 71 Build time: 0:23:18

Core - Full Integration Build 1036 outcome was SUCCESS using a merge of 479a313abf24ec5a53cee1b58eae93cdb9763643
Summary: Tests passed: 2873, ignored: 374 Build time: 0:49:24

authors: ctomc wildfly-ci


text:
WFLY-9449 SingletonPartitionTestCase can fail intermittently

Jira
https://issues.jboss.org/browse/WFLY-9449

Closing for investigation..

repo: wildfly/wildfly


text:
WFLY-13063: adjust sizes of id cache and confirmation window

Issue: WFLY-13063
id-cache-size value is the same, I just changed it not to be hard-coded on our end
confirmation-window-size changed from 1048576 to 10485760, as this seems to be intended default value as per ARTEMIS-1261

Ok, I'll look into that. What's the difference between ModelFixer and Transformer? They seem to be doing the same thing.

ModelFixer 'fix' the model for tests.
Transformers transform the model for retro compatibility

@bstansberry I removed all the references to ActiveMQDefaultConfiguration but it looks like i missed some other Artemis related default configuration. I've created https://issues.redhat.com/browse/WFLY-13575 to track this

I have refactored the PR.

@ehsavoie gotcha, I've updated the file

/retest

Superseded by #14095

authors: bstansberry ehsavoie michpetrov


text:
[WFLY-15031] Exclude wildfly-transaction-client from the WF Preview e…

…e-feature-pack/common bom import so we don't waste time transforming it and misrepresent how many artifacts need to be transformed
Follow up step for https://issues.redhat.com/browse/WFLY-15031

WildFly Preview runs:
https://ci.wildfly.org/viewLog.html?buildId=267490&buildTypeId=WF_Ee9BootableJarLinuxJdk8
https://ci.wildfly.org/viewLog.html?buildId=267492&buildTypeId=WF_WildFlyPreviewLinuxJdk11
Both passed other than the known and unrelated 2 clustering failures.

Superseded by #14642


text:
[WFLY-18448] Update SmallRye OpenTelemetry to 2.5.1

https://issues.redhat.com/browse/WFLY-18448
Bump SR Otel to 2.5.1
Update modules to reflect upstream changes

LGTM

authors: jamezp jasondlee


text:
[JBQA-12586] added new test case for checking how Container and EJB client behave when exception is thrown

The testcase contains various types of scenarios (described here https://mojo.redhat.com/docs/DOC-1071084) and tests whether client receives appropriate exception (defined by specification).
Contains also reproducer for:
WFLY-4672 Possible trouble of exception cause during error of 2PC
EJBCLIENT-155 RollbackException is not propagated from remote EJB client

Can one of the admins verify this patch?

Windows Build 4812 outcome was FAILURE using a merge of eff3dc3ec2fd3a3cf5bde18ec14a691d7a007c29
Summary: Execution timeout (new); tests passed: 3370, ignored: 194 Build time: 01:40:29

retest this please

authors: dsimko wildfly-ci


text:
Matrix

Hello, we use revolt, and matrix left discord 2 months a go, will be nice to have this also connect a matrix room with a revolt room, I was going to write the bot myself in go or rust but then I saw this, so why reinvent the wheel.

I was working on this, I'm just trying to find a way to implement it without going insane requiring people to host a Matrix homeserver to host Bolt and bloating the codebase, which I'm also rewriting (see #18)

Just an update on this, got a bunch more time recently and I'm finalizing a bunch of stuff in the new version Matrix support will be in. should be released soontm

Another update on this: the Matrix plugin on the dev branch supports basic text but not much else at the moment. I'm going to fix that before release though so stay tuned

Hello, sorry I was away waiting for revolt to get a bit more mature, I see this issue is closed, so it is working now? any instructions in how to make it work? cheers!

oh shoot! this shouldn't've been closed and it seems like something else should've been closed instead. #46 is what actually closes this and that's still a bit far off. i'm so sorry about that!

it seems like i'm unable to change the sidebar to say that this is closed by that since github is annoyed at the non-existence of the dev branch, whoops

Thank you @williamhorning :heart:

see #62

Hello everyone! Any news on this? :crying_cat_face:

hey there @r3k2! #66 needs a bit more work before i can get proper matrix support to work, especially with the following things i still have left to do:

improving error handling
handling changes in the plugin api on the main branch (what will be 0.7.3)
making sure it actually works with my homeserver
adapting it to the new workspaces model introduced in deno 1.45

Thanks for the news @williamhorning to bad this project is not on rust or go, I could have helped instead of asking for updates :(
I made a rss bot for our revolt server but don't want to start from 0 in a full on bridge if there is already a project with half the work done.
As a small way to help, we can offer our revolt server in revolt.chat for testing if needed.

hey @r3k2! i've been considering writing the matrix logic in go for a while now, especially since there're libraries there which work really well. there're a few things that approach would need to have solutions for, though:

reply handling would need to change, as right now, the reply function is on the message interface (which would make it more difficult to hand off to go code)
communication between go and typescript code would need to be possible, ideally in a way that allows you to mix/match other languages later on too
the go code for the plugin should be distributed in a way so that it can be shipped similarly to the other plugins

if you have any ideas, i'd love to hear them!

repo: williamhorning/bolt


text:
electric field force

create a force to simulate the electric field on the charged particles.
will also need to add charge property to the particles

Added a Lorenz force method

repo: williammcmp/Nano-Particle-Playground


text:
fix typos

Hi William :>
Is there a proper way to build/generate the guide?

@tofuness Thanks for these! Much appreciated.
Sorry 😅  the guide was generated with an experimental static site app that I'm building. If you just make the changes in .md files, I can compile them. No worries.
Or if you like, I can send you a link to download the app. It's not very stable yet though.

authors: tofuness williamngan

repo: williamngan/pts


text:
Add JSON dump functionality

Added a script called evtx_dump_json.py that outputs evtx files in JSON format or to a file in a JSON array. Primary use case is for easy ingestion into Pandas DataFrames for graph learning, data science, etc. Updated README with reference to new file.

thank you!

repo: williballenthin/python-evtx


text:
Framework - Routing

This includes login/logout, clicking triggers redirects to available games, My Games and account page.

instead of testing with login/logout because that has its own workflow, the front end has basic front end navigation via links to display routing

authors: emilyseibert morganedmonds

repo: willowtreeapps/wombats-web-client


text:
Implement Testing

Implement testing using Selenium

Implement testing using Selenium

Blaise has finished the testing.

repo: willsephton/devops


text:
Move skip_ansible_lint check into get_normalized_tasks()

It is possible we to skip invalid tasks, becase they are custom and
the local environment may not have the plugins installed properly. As
a result, move the skip check into get_normalized_tasks() and skip the
norimalize process if skip_ansible_lint exists.
Signed-off-by: Paul Belanger pabelanger@redhat.com

3.4.12 tagged and pushed to pypi

Thanks, it seems pypi didn't update properly. https://pypi.python.org/pypi/ansible-lint/3.4.12

My new ~/.pypirc wasn't correct - it was pointing at www.python.org/pypi and telling me it was all 200 OK - when I changed it to pypi.python.org/pypi, the upload seems to have taken.

authors: pabelanger willthames

repo: willthames/ansible-lint


text:
Add a field for national ID number

Include a field for things like national ID numbers, social security numbers.
Such numbers uniquely identify people in many countries and can assist emergency services to search national databases of medical records, addresses, telephone numbers more easily to find contacts, allergy, medication, etc. (with the person's consent)
In some countries this is regarded as secret information, in others not.  For instance it is not regarded as sensitive information in Norway, see https://www.datatilsynet.no/rettigheter-og-plikter/personopplysninger/fodselsnummer/

this could prob be added to #25 as a first pass until we come up with solid ux rules for adding new fields

authors: kwhitefoot wilrnh


text:
Should we train the prior model with n_cond_frames>=1 for BAIR Pushing dataset?

I noticed the generated samples shown on the paper for BAIR Pushing dataset are conditioned on a single frame, so I am wondering if we should also train the prior model conditioning on a single frame. However, I saw the default setting for  n_cond_frames is 0 for BAIR Pushing dataset from another repo: VideoGPT-Paper. Also, I'd like to know if the comparison with other methods on BAIR Pushing Dataset is also based on single-frame conditioning.

Yes, for purposes of fair comparisons with prior work, you should train with --n_cond_frames 1. All tables / results in the paper related to BAIR are under the single-conditional setting.


text:
feat: nns resolver

As part of the initiatives to add NNS resolving to different apps in the ecosystem, this pull request added NNS resolving using contract calls.
The original plan was to use the native getEnsName method in viem to resolve NNS names with a drop-in replacement of the NNS resolver address, but unexpected errors arise. Hence, the contract call method from nnsprotocol/nns-resolver-demo is used instead.
The contract call method is definitely less efficient, but it will always fall back to the original resolved ENS name from Basement while loading or if NNS doesn't exist.

build failed, please run yarn lint:fix

@wilsoncusack I have performed lint fix and also refactor the function into lib/nns.ts. Please review.

gm @wilsoncusack
Let us know if it's everything okay or if other changes are needed. Let's make it happen!

authors: nns-dev socathie wilsoncusack