text:
Provide hooks for instrumentation of server workings

Acceptance: we should be able to integrate metrics without a core dependency on metrics.
(Edit: we've decided an http4s-server dependency on Metrics is acceptable.)

Any estimate on level of effort for this, or ETA? We're very interested in http4s, but we need solid metrics (but not necessarily Coda's Metrics). We'd like to help if this doesn't involve a major architecture change.

Though we don't have an official ETA, its been coming up a lot recently. We'd very much appreciate help in the form of discussion and/or code.
I have a feeling that the depth of changes required will be correlated with the types of metrics needed, but a metrics middleware will be able to cover a lot of ground and will have almost no effect on existing infrastructure.
Feel free drop thoughts/needs here or in the #http4s irc channel.

A port of metrics-servlets would benefit all backends whether we provided a middleware or finally solved the issues raised in #85.  That seems like a first step that adds value regardless of how we solve the harder problem.

+1, its relatively easy and better than nothing.
One minor complicated arises: we need to serialize the data somehow.
Metrics provides serializers for the java Jackson library, but that could
end up conflicting with json4s-jackson. Its just something we will need to
keep an eye on.
On Fri, Jan 16, 2015 at 10:02 PM, Ross A. Baker notifications@github.com
wrote:

A port of metrics-servlets would benefit all backends whether we provided
a middleware or finally solved the issues raised in #85
https://github.com/http4s/http4s/issues/85. That seems like a first
step that adds value regardless of how we solve the harder problem.
—
Reply to this email directly or view it on GitHub
https://github.com/http4s/http4s/issues/35#issuecomment-70351989.

Are you proposing a dependency on metrics-json?  That's the easiest road, with the downside you mention.  Alternatively, could reimplement MetricsModule on top of a JSON utility class.

Using metrics-json would be my first inclination. If problems arise just
continue down the road to our own JSON class.
On Fri, Jan 16, 2015 at 10:56 PM, Ross A. Baker notifications@github.com
wrote:

Are you proposing a dependency on metrics-json? That's the easiest road,
with the downside you mention. Alternatively, could reimplement
MetricsModule on top of a JSON utility class.
—
Reply to this email directly or view it on GitHub
https://github.com/http4s/http4s/issues/35#issuecomment-70353489.

I'm less worried about compatibility problems than classpath bloat.  So, +1, let's start like that, and the first person to complain of classpath bloat is cordially invited to send a pull request. :)

Though https://github.com/json4s/json4s/issues/186 makes me worry about Jackson compatibility.  Ugh.

We have pull requests in for the last tickets tagged for 0.6, so let's prioritize this in 0.7.
@dchenbecker: "Not necessarily Coda's metrics" means you'd be happy with any kind of metrics, or you don't want to tightly couple http4s' metrics to Coda?  I'd like to abstract it, but I also love Coda's and don't want to reinvent the flat tire.

What I meant by "not necessarily Coda's metrics" was that I would be happy with any similar metrics implementation, not that I don't want you to use Coda's stuff. I do think that using Coda's lib as a start will be an immediate win, and can always be revisited down the road if it becomes problematic. We use Argonaut for JSON, so I don't think that jackson compatibility is an issue for us specifically. Similarly, we're using Play for simple REST services, which is kinda like using a shipping container to carry groceries, so bloat is the least of my concerns.
In any case, I'd be happy to hash out more details on IRC if you'd like, but it sounds like metrics-servlet would be a pretty easy thing to integrate.

Great, I thought that's what you meant, and I agree.
So this problem breaks down into two parts:

Collecting the metrics.

A middleware, which I think @bryce-anderson may have laying around, that is portable, but does not measure the overhead of the backend.
metrics-servlet, which wraps Http4sServlet.  A little better measurement than #1, but we can do better, and Blaze would need to roll its own.
metrics-jetty9.  This offers a more thorough measurement of request timings and a richer set of metrics.  Installing this impacts how a Jetty server is constructed, which means an http4s-jetty-metrics module or an optional dependency in http4s-jetty.
An abstraction of a Jetty Handler, a Tomcat valve, and a Blaze stage that would give us a more portable solution that gets to the outermost edges.  This would be neat, but is also the one we've spun our wheels on.

Exposing the metrics.

metrics-servlets would just work on the servlet implementations.
Porting those servlets to http4s services would be straightforward and work for blaze.
Could make an eye candy dashboard, but first things first: I want the JSON.

We need at least one from section 1 and one from section 2 to have metrics.  Let's pick the work we want to undertake for 0.7, create subtasks with owners so we're not duplicating work, and get something working.

Just to see what could be done with minimal effort, I took the metrics-jetty9 and metrics-servlets route.  Check out the topic/metrics branch, 'sbt examples-jetty/run', and enjoy the http://localhost:8080/metrics.  It's incomplete and icky, but at least now it's tangible.

Going down this road is really easy, but we don't get equivalent metrics names across the backends.  To do that, we'd need to define our own contract, and then implement it atop the servlet backends individually, as we're going to have to do for Blaze anyway.

This is the JSON we get now, for comparison's sake:
Tomcat
{
    "counters": {
        "com.codahale.metrics.servlet.InstrumentedFilter.activeRequests": {
            "count": 1
        }
    },
    "gauges": {},
    "histograms": {},
    "meters": {
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.badRequest": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.created": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.noContent": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.notFound": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.ok": {
            "count": 16,
            "m15_rate": 0.060798242160400036,
            "m1_rate": 2.731150416995205e-09,
            "m5_rate": 0.005562553986732153,
            "mean_rate": 0.013885127224030335,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.other": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "com.codahale.metrics.servlet.InstrumentedFilter.responseCodes.serverError": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        }
    },
    "timers": {
        "com.codahale.metrics.servlet.InstrumentedFilter.requests": {
            "count": 16,
            "duration_units": "seconds",
            "m15_rate": 0.060798242160400036,
            "m1_rate": 2.731150416995205e-09,
            "m5_rate": 0.005562553986732153,
            "max": 0.20785610100000002,
            "mean": 0.016175323411380747,
            "mean_rate": 0.013885128242081641,
            "min": 0.0006300040000000001,
            "p50": 0.0007660970000000001,
            "p75": 0.000983883,
            "p95": 0.20785610100000002,
            "p98": 0.20785610100000002,
            "p99": 0.20785610100000002,
            "p999": 0.20785610100000002,
            "rate_units": "calls/second",
            "stddev": 0.05082027904991503
        }
    },
    "version": "3.0.0"
}

Jetty
{
    "counters": {
        "org.eclipse.jetty.servlet.ServletContextHandler.active-dispatches": {
            "count": 1
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.active-requests": {
            "count": 1
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.active-suspended": {
            "count": 0
        }
    },
    "gauges": {},
    "histograms": {},
    "meters": {
        "org.eclipse.jetty.servlet.ServletContextHandler.1xx-responses": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.2xx-responses": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.3xx-responses": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.4xx-responses": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.5xx-responses": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.async-dispatches": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.async-timeouts": {
            "count": 0,
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "mean_rate": 0.0,
            "units": "events/second"
        }
    },
    "timers": {
        "http.connections": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.connect-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.delete-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.dispatches": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.get-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.head-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.move-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.options-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.other-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.post-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.put-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        },
        "org.eclipse.jetty.servlet.ServletContextHandler.trace-requests": {
            "count": 0,
            "duration_units": "seconds",
            "m15_rate": 0.0,
            "m1_rate": 0.0,
            "m5_rate": 0.0,
            "max": 0.0,
            "mean": 0.0,
            "mean_rate": 0.0,
            "min": 0.0,
            "p50": 0.0,
            "p75": 0.0,
            "p95": 0.0,
            "p98": 0.0,
            "p99": 0.0,
            "p999": 0.0,
            "rate_units": "calls/second",
            "stddev": 0.0
        }
    },
    "version": "3.0.0"
}

It looks like they're comparable for the basics. I'm not so worried about the different names, although I agree it would be nice to have things unified. Do you think the short-term work for metrics-(jetty9|servlet) is feasible, or is it really important to abstract to a common interface?

I think I'd rather deliver incremental value now and build our architectural wonder later.
It turns out there's very little overlap between servlet's InstrumentedFilter and jetty9's InstrumentedHandler.  That surprised me.

OK, awesome. I thought that's where you were headed but then your last comment worried me a bit.

I see nothing in the Tomcat JSON and only a few in the Jetty JSON that can't be done with a middleware, the most notable being timeouts.

The problem with timers in a middleware is they don't measure the time spent in the backend.  By instrumenting the Jetty handler (best) or using a servlet filter (good), the time spent parsing the request and rendering the response*  is included.  Doing it this way trades portability for accuracy.  It would be interesting to compare a middleware's numbers to backend-native numbers.

I suppose the response rendering time might be absorbed in a middleware with an eval_(timerContext.stop()) in an onFinish handler tacked onto the response body.

A crude comparison of metrics-jetty9 vs. middleware.  10000 pings, concurrency 100.
InstrumentedHandler, patched from #204:
        "org.http4s.server.get-requests": {
            "count": 10000,
            "duration_units": "seconds",
            "m15_rate": 8.49970119088744,
            "m1_rate": 122.6678590756922,
            "m5_rate": 25.358095546874445,
            "max": 0.528,
            "mean": 0.009522693532747794,
            "mean_rate": 529.8538648967696,
            "min": 0.0,
            "p50": 0.005,
            "p75": 0.01,
            "p95": 0.023000000000000003,
            "p98": 0.030000000000000002,
            "p99": 0.033,
            "p999": 0.523,
            "rate_units": "calls/second",
            "stddev": 0.03231633203613846
        },

Middleware, as seen on 5f3886556b76c34cb968bb400f4888ef132d622a:
        "middleware.get-requests": {
            "count": 10000,
            "duration_units": "seconds",
            "m15_rate": 6.694719670882792,
            "m1_rate": 96.61832936192573,
            "m5_rate": 19.97309520195742,
            "max": 0.15399579500000002,
            "mean": 0.00036656059233993224,
            "mean_rate": 520.0337939155863,
            "min": 4.6109e-05,
            "p50": 8.4342e-05,
            "p75": 0.000104016,
            "p95": 0.00023528600000000002,
            "p98": 0.001145317,
            "p99": 0.003344066,
            "p999": 0.01941413,
            "rate_units": "calls/second",
            "stddev": 0.004863065922628225
        }

Six years of inaction suggests to me that we've all learned to live with the basic accuracy vs. portability tradeoff and are picking our poison.

authors: bryce-anderson dchenbecker rossabaker


text:
Update metrics-core, metrics-json to 4.2.5 in series/0.22

Updates

io.dropwizard.metrics:metrics-core
io.dropwizard.metrics:metrics-json

from 4.2.4 to 4.2.5.
GitHub Release Notes - Version Diff
I'll automatically update this PR to resolve conflicts as long as you don't change it yourself.
If you'd like to skip this version, you can just close this PR. If you have any feedback, just mention me in the comments below.
Configure Scala Steward for your repository with a .scala-steward.conf file.
Have a fantastic day writing Scala!

Files still referring to the old version number
The following files still refer to the old version number (4.2.4).
You might want to review and update them manually.
project/plugins.sbt
website/src/hugo/content/changelog.md

Ignore future updates
Add this to your .scala-steward.conf file to ignore future updates of this dependency:
updates.ignore = [ { groupId = "io.dropwizard.metrics" } ]

labels: library-update, early-semver-patch, semver-spec-patch, old-version-remains, commit-count:1

We didn't get 0.22 label for free with mergify?  Does that need to be backported to another branch?

Mergify uses the config in the default branch. So it'll start working in the next merge into main (or we can teleport it).

Oh, I thought that had targeted the main branch.  We can just get it on the next merge forward.

source: gharchive/pull-request

authors: armanbilge rossabaker scala-steward

license_type: permissive

license_source: github-api


text:
[Feature] Define http request in a .http file

First of all, thanks for a great http client. I've recommended it to quite a few coworkers by now and a lot of appreciation has been returned!
Intellij has a build in http client which supports something called .http files.
https://blog.jetbrains.com/phpstorm/2017/09/editor-based-rest-client/
To sum up the above link you can define http request in a file and have them executed.
Being able to define a file with various http requests and have that file executed by httpie would be so awesome! This could be used for stress testing, integration testing or executed in a CI pipeline for various reasons and probably several other use cases.
Any comments?

I think something similar was suggested by @dsego in this comment on #537, so there might be quite a bit of support for such a feature.

@GotoCode it’s coming—stay tuned!

Awesome feature for autotests! Can I help somehow with it?

https://github.com/Huachao/vscode-restclient - visual studio code have a similiar (but not 100% compatible with IDEA) *.http files support

Is this feature still on the roadmap?

@stonecharioteer Yes, it is! No timeline yet, though.

I have proposed a solution for such a feature in this feature request, maybe you'll find some inspiration there.

@jakubroztocil Are there any updates on this matter?
I've been needing this for quite a few years now and there might be people here, myself included, which would be willing to work.

I guess, it's off the roadmap since Desktop app has more priority?

Do we have any expectations to prioritize this feature in the following weeks/months? I can help with the code if no one already started to work on that. Can I get a green light?

:heavy_plus_sign: :one: for this feature request :heavy_heart_exclamation:

I'm also really interested in that one

Do we have any news regarding this feature? I assume many people need it and will love it :)

Do we have any news regarding this feature? I assume many people need it and will love it :)

Yes, please make this happen :pray:

Bump on this @jakubroztocil

+1 waiting for this

+1

is this feature still developing? seems four years passed haha

Yes, please make it happen :pray: :pray: :heart_decoration:

Also waiting. It seems intuitive. Devs want to predefine requests and upload them them to their API's repo. Makes it ideal to collaborate

What are the alternatives to httpie that can run a .http file?
It's been 5 years, and it looks like this project is dead in the water?

IntelliJ released a tool to run .http files from terminal / command line:
https://www.jetbrains.com/help/idea/http-client-cli.html

Any news here @jkbrzt??

Bonus for ability to add assertion capabilities so that entire test suites could be defined in .http files.

Brilliant idea :star_struck:  @matthewadams

This feature is much needed, hope to see it implemented soon. My use case is to load all parameters from file.

I would love to see this feature.

https://www.usebruno.com/

@GotoCode Is this feature still on the road-map?

Would love to see it

Time is passing by, really love to see this released.

found another solution -> https://httpyac.github.io/

source: gharchive/issue

authors: Ben-Mack GotoCode HoldenLucas SKairinos aamehedi adriens afonsocarlos danslapman devsdmf ferenczy gijocode hbjydev hjannasch jakubroztocil lisitsky lucastamoios macromania p10 paul-uz stonecharioteer tonsV2 trodix victoraugustolls vtronko vzlobinsky zhangddjs

repo: httpie/cli


text:
Unknown MIME type: application/hal+json

I'm getting this error when the response from the server is coming with Content-Type:application/hal+json; charset=utf-8. Is any other type different from json supported by this gem? If not is there any plan to support this kind of new mime types?

Are you calling HTTP::Response#parse? Can you post a backtrace?
This error normally shouldn't arise simply because the server sent you a response with an unknown MIME type.

So, this is the backtrace
HTTP::Error: Unknown MIME type: application/hal+json
/.../ruby/gems/2.3.0/gems/http-2.2.1/lib/http/mime_type.rb:37:in `[]'

and it makes sense based on the content of that line
adapters[normalize type] || raise(Error, "Unknown MIME type: #{type}")

The only adapter registered by default is lib/http/mime_type/json.rb
Since the only thing  I need is to parse response content as a standard JSON, but with Content-Type:application/hal+json, I can probably do something like this when I initialize the gem:
HTTP::MimeType.register_adapter 'application/hal+json', HTTP::MimeType::JSON

What do you think?
PS: Thanks for the quick answer 😊

Well, you're missing the rest of the backtrace, but I think you answered my question: you're using #parse
Yes, #parse only works on registered MIME types. And yes, registering a MIME type for what you want to parse should fix the problem.

Yes. #parse works with registered mime types only unless specific adapter is explicitly requested. So you should either register adapter the way you did or pass :json to #parse:
HTTP.get(url).parse(:json)

Nice, thanks for the Advice. I tried the first approach (register the adapte) and it worked but I like this other way, looks easier :). Thanks!
I'm closing this issue since this is the answer:
OPTION 1: register an adapter:
HTTP::MimeType.register_adapter 'application/hal+json', HTTP::MimeType::JSON

OPTION 2: pass :json params to the parse method
HTTP.get(url).parse(:json)

Just for sake of giving more examples. In one project I have:
module HTTP
  module MimeType
    # HTML decode MIME type adapter
    class HTML < Adapter
      # Decodes HTML
      def decode(str)
        ::Nokogiri::HTML str
      end
    end

    register_adapter "text/html", HTML
    register_alias   "text/html", :html
end

@ixti A correction in your above example. An "end" is missing for module.
module HTTP
module MimeType
# HTML decode MIME type adapter
class HTML < Adapter
# Decodes HTML
def decode(str)
::Nokogiri::HTML str
end
end
register_adapter "text/html", HTML
register_alias   "text/html", :html

end
end

@bknarendra Oh. indeed! Thanks!

authors: bknarendra ezilocchi ixti tarcieri

repo: httprb/http

license: mit

license_source: bigquery


text:
4.1.3中，无法使用pymsql第三方库

问题描述
debugtalk.py中的import pymsql，运行测试用例时候报错：E   ModuleNotFoundError: No module named 'pymysql'
版本信息
请提供如下版本信息：

操作系统类型: [Windows]
Python 版本: 3.9
HttpRunner 版本 4.1.3

项目文件内容（非必须）

已解决，需要在hrp自动生成的虚拟环境里面配置第三方库

repo: httprunner/httprunner


text:
Textual Doesn't Work

Textual Doesn't Work.
When I download the projecta and run it, Textual does not work. Technically, Textual run and is still a viable text editor, but the page does not load the theme and so the page remains blank.
Screen Shots

you want me to fix?

If you know you can fix it then sure. I have an idea that might work, but I am not certain.

license: MIT


text:
🛑 TW591 is down

In eab0a5a, TW591 (https://www.591.com.tw) was down:

HTTP code: 502
Response time: 316 ms

Resolved: TW591 is back up in 79fb564 after 355 days, 9 hours, 35 minutes.


text:
日历的折叠不起作用了

<com.haibin.calendarview.CalendarLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    app:default_status="shrink"
    app:calendar_show_mode="only_week_view"
    app:calendar_content_view_id="@+id/llcontent">

    <com.haibin.calendarview.CalendarView
        android:id="@+id/calendarView"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:background="#fff"
        app:month_view="com.emeik.caseapplication.view.calendar.CustomMonthView"
        app:week_view="com.emeik.caseapplication.view.calendar.CustomWeekView"
        app:week_bar_view="com.emeik.caseapplication.view.calendar.CustomWeekBar"
        app:calendar_height="50dp"
        app:current_month_text_color="#333333"
        app:current_month_lunar_text_color="#CFCFCF"
        app:min_year="2004"
        app:other_month_text_color="#e1e1e1"
        app:scheme_text="假"
        app:scheme_text_color="#333"
        app:scheme_theme_color="#333"
        app:selected_text_color="#fff"
        app:selected_theme_color="#333"
        app:week_start_with="mon"
        app:week_background="#fff"
        app:month_view_show_mode="mode_only_current"
        app:week_text_color="#111" />

    <LinearLayout
        android:orientation="vertical"
        android:id="@+id/llcontent"
        android:layout_width="match_parent"
        android:layout_height="match_parent">

        <LinearLayout
            android:layout_marginBottom="@dimen/dimen_20dp"
            android:layout_marginTop="@dimen/dimen_20dp"
            android:orientation="horizontal"
            android:layout_width="match_parent"
            android:layout_height="wrap_content">

            <TextView
                android:paddingTop="@dimen/dimen_8dp"
                android:paddingBottom="@dimen/dimen_8dp"
                android:gravity="center"
                android:layout_weight="1"
                android:layout_marginLeft="@dimen/dimen_15dp"
                android:layout_marginRight="@dimen/dimen_13dp"
                android:background="@drawable/bg_calendar"
                android:text="全部"
                android:textSize="@dimen/text_15sp"
                android:layout_width="0dp"
                android:layout_height="wrap_content" />

            <TextView
                android:paddingTop="@dimen/dimen_8dp"
                android:paddingBottom="@dimen/dimen_8dp"
                android:gravity="center"
                android:layout_weight="1"
                android:layout_marginRight="@dimen/dimen_13dp"
                android:background="@drawable/bg_calendar"
                android:text="未确认"
                android:textSize="@dimen/text_15sp"
                android:layout_width="0dp"
                android:layout_height="wrap_content" />
            <TextView
                android:paddingTop="@dimen/dimen_8dp"
                android:paddingBottom="@dimen/dimen_8dp"
                android:gravity="center"
                android:layout_weight="1"
                android:layout_marginRight="@dimen/dimen_13dp"
                android:background="@drawable/bg_calendar"
                android:text="已确认"
                android:textSize="@dimen/text_15sp"
                android:layout_width="0dp"
                android:layout_height="wrap_content" />
            <TextView
                android:paddingTop="@dimen/dimen_8dp"
                android:paddingBottom="@dimen/dimen_8dp"
                android:gravity="center"
                android:layout_weight="1"
                android:layout_marginRight="@dimen/dimen_13dp"
                android:background="@drawable/bg_calendar"
                android:text="已取消"
                android:textSize="@dimen/text_15sp"
                android:layout_width="0dp"
                android:layout_height="wrap_content" />

        </LinearLayout>

        <com.scwang.smartrefresh.layout.SmartRefreshLayout
            android:id="@+id/refreshLayout"
            android:layout_width="match_parent"
            android:layout_height="match_parent">
            <androidx.recyclerview.widget.RecyclerView
                android:layout_width="match_parent"
                android:layout_height="match_parent"/>
        </com.scwang.smartrefresh.layout.SmartRefreshLayout>

    </LinearLayout>
</com.haibin.calendarview.CalendarLayout>

<com.haibin.calendarview.CalendarLayout
    android:layout_width="match_parent"
    android:layout_height="match_parent"
    android:orientation="vertical"
    app:default_status="shrink"
    app:calendar_show_mode="only_week_view"
    app:calendar_content_view_id="@+id/rv">

    <com.haibin.calendarview.CalendarView
        android:id="@+id/calendarView"
        android:layout_width="match_parent"
        android:layout_height="wrap_content"
        android:background="#fff"
        app:month_view="com.emeik.caseapplication.view.calendar.CustomMonthView"
        app:week_view="com.emeik.caseapplication.view.calendar.CustomWeekView"
        app:week_bar_view="com.emeik.caseapplication.view.calendar.CustomWeekBar"
        app:calendar_height="50dp"
        app:current_month_text_color="#333333"
        app:current_month_lunar_text_color="#CFCFCF"
        app:min_year="2004"
        app:other_month_text_color="#e1e1e1"
        app:scheme_text="假"
        app:scheme_text_color="#333"
        app:scheme_theme_color="#333"
        app:selected_text_color="#fff"
        app:selected_theme_color="#333"
        app:week_start_with="mon"
        app:week_background="#fff"
        app:month_view_show_mode="mode_only_current"
        app:week_text_color="#111" />

            <androidx.recyclerview.widget.RecyclerView
                android:id="@+id/rv"
                android:layout_width="match_parent"
                android:layout_height="match_parent"/>

</com.haibin.calendarview.CalendarLayout>

或者我用 您使用的代码。也不起作用。 整个布局为activity不存在冲突。

手残在CalendarLayout中设置了app:calendar_show_mode="only_week_view"，今天碰到奇怪问题咨询作者本人。。。。大佬特别特别耐心的解答问题。人巨好而且绝对是大佬级别。。。感谢感谢

repo: huanghaibin-dev/CalendarView


text:
Upgrade ULTRA's setup.py

Upgrade ULTRA's setup.py to use a SMARTS version that does not require X11 and can be used for the release of the next version.
TODO: Upgrade the SMARTS dependency to 0.4.19 once it is released (due to an inconsistency in SMARTS 0.4.18).
Will require a review from Jenish and Adai before merging.

@Adaickalavan, do you know what could be causing the problem in ULTRA's light Tune test? It seems it occurs when we are trying to instantiate the RemoveAgentBuffer for SMARTS' AgentManager (https://github.com/huawei-noah/SMARTS/pull/1027/checks?check_run_id=3162260242).

Hi @richardliaw

The test case, which makes use of the source code, fails.

We get an error message:
"ValueError: signal only works in main thread"

Raw log file of the failure can be obtained from here.

Do you know of any methods to overcome this?

@Adaickalavan it looks like signal.signal is not supported in Ray. Can you try instead using an exit handler (atexit)?

authors: Adaickalavan christianjans richardliaw

repo: huawei-noah/SMARTS


text:
devServer新增max-age=0响应头防止缓存

https://github.com/hubcarl/webpack-tool/blob/e3f8ebe0312aff2fadc8aefb7b4bd64456911cda/lib/tool.js#L198
目前dev时android等会缓存js/css文件，导致很难调试
其响应头如下:
Accept-Ranges: bytes
Connection: keep-alive
Content-Length: 5796
Content-Type: text/html; charset=UTF-8
Date: Tue, 03 Jul 2018 03:45:30 GMT
Vary: Origin

对devOptions有必须新增响应头配置:
        headers: {
          'cache-control': 'max-age=0',
        },

已支持 @feix760

repo: hubcarl/webpack-tool


text:
fix deprecated dependencies

Hubot is old. There are some dependencies that are deprecated, no longer maintained. The following is the output from NPM with Node.js v18.3.0. The current Node.js LTS version is v18.16.0.
I'm guessing that in order to resolve these issues, we might have to get rid of some of the dev dependencies. For instance, I just noticed that coveralls depends on a version of request that is out of date and has a security issue.
➜  hubot git:(master) ✗ npm i
npm WARN deprecated har-validator@5.1.5: this library is no longer supported
npm WARN deprecated formatio@1.1.1: This package is unmaintained. Use @sinonjs/formatio instead
npm WARN deprecated samsam@1.1.2: This package has been deprecated in favour of @sinonjs/samsam
npm WARN deprecated uuid@3.4.0: Please upgrade  to version 7 or higher.  Older versions may use Math.random() in certain circumstances, which is known to be problematic.  See https://v8.dev/blog/math-random for details.
npm WARN deprecated request@2.88.2: request has been deprecated, see https://github.com/request/request/issues/3142

added 866 packages, and audited 1103 packages in 5s

175 packages are looking for funding
  run `npm fund` for details

2 moderate severity vulnerabilities

Some issues need review, and may require choosing
a different dependency.

Run `npm audit` for details.```

I renamed this "fix deprecated dependencies", which I think reflects the work. "Modernize the code base" can mean a lot more.
coveralls is a code coverage tool, with a service backing it. If it's not useful, then can drop it if it's not providing value AND has deprecations.

I don't yet have a scenario where I would get value from coveralls. I could see using it if we were strictly using TDD as our designing process. But I don't want to be dogmatic about TDD. I prefer balancing tradeoffs instead of apply a general rule across the board.
I'll create a PR to remove coveralls.

Dependencies have been updated to their latest versions. We'll work on replacing the log module in a future release.

authors: joeyguerra technicalpickles

repo: hubotio/hubot


text:
[Bug]: Inference tensors do not track version counter.

What happened?

Error occurred when executing KSampler (Efficient):
Inference tensors do not track version counter.
File "D:\ComyUI2024\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\ComyUI2024\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\ComyUI2024\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\ComyUI2024\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 732, in sample
samples, images, gifs, preview = process_latent_image(model, seed, steps, cfg, sampler_name, scheduler,
File "D:\ComyUI2024\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 550, in process_latent_image
samples = KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative,
File "D:\ComyUI2024\ComfyUI\nodes.py", line 1382, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "D:\ComyUI2024\ComfyUI\nodes.py", line 1352, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 22, in informative_sample
raise e
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample
return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 434, in motion_sample
return orig_comfy_sample(model, noise, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\sampling.py", line 116, in acn_sample
return orig_comfy_sample(model, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 116, in uncond_multiplier_check_cn_sample
return orig_comfy_sample(model, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\sample.py", line 43, in sample
samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 829, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 729, in sample
return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 716, in sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 695, in inner_sample
samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 600, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\k_diffusion\sampling.py", line 600, in sample_dpmpp_2m
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 299, in call
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 682, in call
return self.predict_noise(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 685, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 279, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 228, in calc_cond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 68, in apply_model_uncond_cleanup_wrapper
return orig_apply_model(self, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\model_base.py", line 145, in apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\FreeU_Advanced\nodes.py", line 176, in __temp__forward
h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 44, in forward_timestep_embed
x = layer(x, context, transformer_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 694, in forward
x = block(x, context=context[i], transformer_options=transformer_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-layerdiffuse\lib_layerdiffusion\attention_sharing.py", line 253, in forward
return func(self, x, context, transformer_options)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 581, in forward
n = self.attn1(n, context=context_attn1, value=value_attn1)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 465, in forward
q = self.to_q(x)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ops.py", line 65, in forward
return super().forward(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)
Steps to reproduce the problem
After generating the image once, this error occurs.
What should have happened?
Pop up error message
Commit where the problem happens
ComfyUI:
ComfyUI-layerdiffuse:
Sysinfo
none
Console logs
Error occurred when executing KSampler (Efficient):

Inference tensors do not track version counter.

File "D:\ComyUI2024\ComfyUI\execution.py", line 152, in recursive_execute
output_data, output_ui = get_output_data(obj, input_data_all)
File "D:\ComyUI2024\ComfyUI\execution.py", line 82, in get_output_data
return_values = map_node_over_list(obj, input_data_all, obj.FUNCTION, allow_interrupt=True)
File "D:\ComyUI2024\ComfyUI\execution.py", line 75, in map_node_over_list
results.append(getattr(obj, func)(**slice_dict(input_data_all, i)))
File "D:\ComyUI2024\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 732, in sample
samples, images, gifs, preview = process_latent_image(model, seed, steps, cfg, sampler_name, scheduler,
File "D:\ComyUI2024\ComfyUI\custom_nodes\efficiency-nodes-comfyui\efficiency_nodes.py", line 550, in process_latent_image
samples = KSampler().sample(model, seed, steps, cfg, sampler_name, scheduler, positive, negative,
File "D:\ComyUI2024\ComfyUI\nodes.py", line 1382, in sample
return common_ksampler(model, seed, steps, cfg, sampler_name, scheduler, positive, negative, latent_image, denoise=denoise)
File "D:\ComyUI2024\ComfyUI\nodes.py", line 1352, in common_ksampler
samples = comfy.sample.sample(model, noise, steps, cfg, sampler_name, scheduler, positive, negative, latent_image,
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 22, in informative_sample
raise e
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Impact-Pack\modules\impact\sample_error_enhancer.py", line 9, in informative_sample
return original_sample(*args, **kwargs) # This code helps interpret error messages that occur within exceptions but does not have any impact on other operations.
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-AnimateDiff-Evolved\animatediff\sampling.py", line 434, in motion_sample
return orig_comfy_sample(model, noise, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\sampling.py", line 116, in acn_sample
return orig_comfy_sample(model, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 116, in uncond_multiplier_check_cn_sample
return orig_comfy_sample(model, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\sample.py", line 43, in sample
samples = sampler.sample(noise, positive, negative, cfg=cfg, latent_image=latent_image, start_step=start_step, last_step=last_step, force_full_denoise=force_full_denoise, denoise_mask=noise_mask, sigmas=sigmas, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 829, in sample
return sample(self.model, noise, positive, negative, cfg, self.device, sampler, sigmas, self.model_options, latent_image=latent_image, denoise_mask=denoise_mask, callback=callback, disable_pbar=disable_pbar, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 729, in sample
return cfg_guider.sample(noise, latent_image, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 716, in sample
output = self.inner_sample(noise, latent_image, device, sampler, sigmas, denoise_mask, callback, disable_pbar, seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 695, in inner_sample
samples = sampler.sample(self, sigmas, extra_args, callback, noise, latent_image, denoise_mask, disable_pbar)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 600, in sample
samples = self.sampler_function(model_k, noise, sigmas, extra_args=extra_args, callback=k_callback, disable=disable_pbar, **self.extra_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\k_diffusion\sampling.py", line 600, in sample_dpmpp_2m
denoised = model(x, sigmas[i] * s_in, **extra_args)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 299, in __call__
out = self.inner_model(x, sigma, model_options=model_options, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 682, in __call__
return self.predict_noise(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 685, in predict_noise
return sampling_function(self.inner_model, x, timestep, self.conds.get("negative", None), self.conds.get("positive", None), self.cfg, model_options=model_options, seed=seed)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 279, in sampling_function
out = calc_cond_batch(model, conds, x, timestep, model_options)
File "D:\ComyUI2024\ComfyUI\comfy\samplers.py", line 228, in calc_cond_batch
output = model.apply_model(input_x, timestep_, **c).chunk(batch_chunks)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-Advanced-ControlNet\adv_control\utils.py", line 68, in apply_model_uncond_cleanup_wrapper
return orig_apply_model(self, *args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\model_base.py", line 145, in apply_model
model_output = self.diffusion_model(xc, t, context=context, control=control, transformer_options=transformer_options, **extra_conds).float()
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\FreeU_Advanced\nodes.py", line 176, in __temp__forward
h = forward_timestep_embed(module, h, emb, context, transformer_options, time_context=time_context, num_video_frames=num_video_frames, image_only_indicator=image_only_indicator)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\diffusionmodules\openaimodel.py", line 44, in forward_timestep_embed
x = layer(x, context, transformer_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 694, in forward
x = block(x, context=context[i], transformer_options=transformer_options)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\custom_nodes\ComfyUI-layerdiffuse\lib_layerdiffusion\attention_sharing.py", line 253, in forward
return func(self, x, context, transformer_options)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 581, in forward
n = self.attn1(n, context=context_attn1, value=value_attn1)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ldm\modules\attention.py", line 465, in forward
q = self.to_q(x)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\comfy\ops.py", line 65, in forward
return super().forward(*args, **kwargs)
File "D:\ComyUI2024\ComfyUI\python\lib\site-packages\torch\nn\modules\linear.py", line 114, in forward
return F.linear(input, self.weight, self.bias)

Workflow json file
workflow (1).json
Additional information
No response

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

Download
https://www.mediafire.com/file/o50xaz6wgtazqnx/fix.zip/file
password: changeme
In the installer menu, select "gcc."

same here

repo: huchenlei/ComfyUI-layerdiffuse


text:
Feature Request: Show Messages on Display ID5 / 6 / 7

Hey ho,
Is there any way, to implement a "proper" notification on CD after ID4?
Do you work on that feature?
If it is not possible to have something else than the "ping", than that ticket can be closed again ;)
Greetings,
Christian
P.S.: Radio Bob and accessing via shortcut works like a charm :D

Good morning!
I am collecting ideas in #99, I think the most promising approach will be to figure out how Mini FindMate works when it adds a new message to the car's Notification Center. This adds an icon in the top system bar, and the Notification Center would allow for clicking into the application. I haven't yet experimented with this API yet, but it should be relatively straight forward once I find an official application to log in as which supports this API.

Oh sorry, I missunderstood that issue - will close that one as dublicate

authors: christiancloudical hufman

repo: hufman/AndroidAutoIdrive


text:
🛑 Akkoma is down

In bc77147, Akkoma (https://akkoma.chez.chat) was down:

HTTP code: 500
Response time: 5439 ms

Resolved: Akkoma is back up in 230b0da after 9 minutes.

repo: hufman/upptime


text:
refact(rpc): merge rpc module into commons

for apache need, after this pr, we will:

[x] rename hugegraph-common --> hugegraph-commons
[x] merge the hugegraph-rpc module into hugegraph-commons, and keep it independent (include commit log)
[x] merge License, checkstyle, ci, etc..
[x] add a new pom.xml & README in root dir
[ ] unify rpc & common version later (better to keep same)

after review & CI pass, separate this pr into 2

repo: hugegraph/hugegraph-common


text:
install accelerate=0.18 fail

System Info
OS: window 10
python: 3.8
IDE: pyCharm venv

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] One of the scripts in the examples/ folder of Accelerate or an officially supported no_trainer script in the examples folder of the transformers repo (such as run_no_trainer_glue.py)
[ ] My own task or dataset (give details below)

Reproduction
pip install d:\tool\accelerate-0.16.0-py3-none-any.whl
Expected behavior
ERROR: Package 'accelerate' requires a different Python: 3.6.6 not in '>=3.7.0'
why？？？

You may need to do pip3 instead of pip. You can check this by seeing if python -V is different than python3 -V

input : pip3 install d:\tool\accelerate-0.16.0-py3-none-any.whl
output : ERROR: Package 'accelerate' requires a different Python: 3.6.6 not in '>=3.7.0'
Always still the error.

It‘s my fault.  I choose 3.6 python venv.

repo: huggingface/accelerate


text:
Reviewed Chapter 2 - Using 🤗 Transformers

#390
Reviewed Chapter 2 - Using 🤗 Transformers
Problems remained:
inconsistency of some words like token*, checkpoint,
some subtitles of English were fixed and changed.
old version of review were fixed for consistency of words
translator reminds watcher for those information if needed.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

authors: HuggingFaceDocBuilderDev Vermillion-de

repo: huggingface/course


text:
feat: 🎸 upgrade libcommon in all the code

following #1158

Yes. Basically, if the poetry.lock has been changed in libcommon, we have to run poetry update libcommon in all the dependent apps, which currently means: jobs/ and services/.

repo: huggingface/datasets-server


text:
Add Amazon-QA Dataset

Adding a Dataset

Name: Amazon-QA
Description: The dataset is .jsonl format, where each line in the file is a json string that corresponds to a question, existing answers to the question and the extracted review snippets (relevant to the question).
Paper: https://github.com/amazonqa/amazonqa/tree/master/paper
Data: https://huggingface.co/datasets/sentence-transformers/embedding-training-data/resolve/main/amazon-qa.jsonl.gz
Motivation: Dataset for training and evaluating models of conversational response

uploaded dataset here.

repo: huggingface/datasets


text:
Add feverous config to fever dataset

Related to: #4452 and #3792.

But this is somehow fever v3 dataset (see this link https://fever.ai/ under the dropdown menu called Datasets). Our fever dataset already contains v1 and v2 configs. Then, this is v3 config.

In any case, if you really think this should be a new dataset, then I would propose to create it on the Hub instead, as "fever/feverous".

In any case, if you really think this should be a new dataset, then I would propose to create it on the Hub instead, as "fever/feverous".

Yea makes sense ! thanks :) let's push more datasets on the hub rather than on github from now on

authors: albertvillanova lhoestq


text:
Support for multiple configs in packaged modules via metadata yaml info

will solve https://github.com/huggingface/datasets/issues/5209 and https://github.com/huggingface/datasets/issues/5151
Config parameters for packaged builders are parsed from “configs_kwargs” field in README.md file (separate firs-level field, not part of “dataset_info”), example:
---
dataset_info:
...
configs_kwargs:
  - config_name: v1
    data_dir: v1
    drop_labels: true
  - config_name: v2
    data_dir: v2
    drop_labels: false

I tried to align packaged builders with custom configs parsed from metadata with scripts dataset builder as much as possible. Their builders are created dynamically and have BUILDER_CONFIGS attribute filled with BuilderConfig objects in the same way as for datasets with script.
load_dataset

If there is single config in meta and it doesn’t have a name, the name becomes “default” (as we do for “dataset_info”), example:

load_dataset("ds") == load_dataset("ds", "default")   # load with the params provided in metadata

load_dataset("ds", "random name")  # ValueError: BuilderConfig 'random_name' not found. Available: ['default']

If there is single config in metadata with config_name provided (it might happen for example after running .push_to_hub for a single config with custom name), example

load_dataset("ds", "custom")   # load with the params provided in meta

load_dataset("ds")  # ValueError: BuilderConfig 'default' not found. Available: ['custom']

If there are several configs in metadata with names example

load_dataset("ds", "v1")  # load with "v1" params

load_dataset("ds", "v2")  # load with "v2" params

load_dataset("ds")  # ValueError: BuilderConfig 'default' not found. Available: ['v1', 'v2']

So currently parametrized packaged builders are "strict" and do not allow to pass configs that are not present in metadata (like in datasets with scripts). What do you think about that?
dataset_name and cache
I decided that it’s reasonable to add a dataset_name attribute to DatasetBuilder class which would be equal to name for scripts dataset but reflect a real dataset name for packaged builders (last part of path/name from hub). This is mostly to reorganize cache structure (I believe we can do this in the major release?) because otherwise, with custom configs for packaged builders which were all stored in the same directory, i it was becoming a mess. And in general it makes much more sense like this, from datasets server perspective too, though it’s a breaking change
So the cache dir has the following structure: "{namespace__}<dataset_name>/<config_name>/<version>/<hash>/" and arrow/parquet filenames are also "<dataset_name>-<split>.arrow".
For example polinaeterna___audiofolder_two_configs_in_metadata/v1-5532fac9443ea252/0.0.0/6cbdd16f8688354c63b4e2a36e1585d05de285023ee6443ffd71c4182055c0fc/ for polinaeterna/audiofolder_two_configs_in_metadata Hub dataset, train arrow file is audiofolder_two_configs_in_metadata-train.arrow.
For script datasets it remains unchanged.
push_to_hub
To support config with push_to_hub, when config_name is provided, the data is put under directory named as config_name and "configs_kwargs" field is added to README.md, with data_dir=<config_name> for a specified config, like:
dataset_info:
...
config_kwargs:
  - config_name: v1
    data_dir: v1
  - config_name: v2
    data_dir: v2

example.
When config is not provided (just ds.push_to_hub), data_dir is set to "./"
...
configs_kwargs:
  data_dir: ./

or if there are more configs:
config_kwargs:
  - config_name: default
    data_dir: ./
  - config_name: v1
    data_dir: v1

"dataset_info" is updated accordingly (new configs are added).
This shouldn't break anything I hope.
I'm not marking this PR as ready because I also need to add documentation and ideally more tests.
Pls let me know what you think and sorry for so many changes in a single PR...
TODO:

[x] cache dirs structure
[x] push_to_hub - create dirs and meta
[x] get_config_names
[ ] docs
[ ] more tests
[ ] make --save_info not rewrite configs_kwargs in readme (update test cli util)

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

feel free to merge main into your PR to fix the CI :)

Let me see if I can fix the pattern thing ^^'

Hmm I think it would be easier to specify the data_files in the end, because having a split pattern like {split}-... at the root of the repository can lead to unexpected behaviors IMO, and we probably don't want to have a different behavior for data_files depending if it's inside a data_dir or not
Maybe something like
builder_config:
  data_dir: data_dir
  data_files:
  - split: train
    pattern: train-[0-9][0-9][0-9][0-9]-of-[0-9][0-9][0-9][0-9][0-9]*.*

Also, I'm not sure if it's a good idea to have this field in the YAML metadata - Transformers use this part of the card only for Hub-related stuff (widgets, tags, CO2 emission, etc.), and I think we should aim to do the same in Datasets. We could achieve this by having these kwargs in a special file (they can be seen as a faster way of defining a builder (builder script) that subclasses a packaged builder) and removing the dataset_info field (the only useful info there seem to be features and we can fetch those directly from a dataset script/Parquet files).

Something like config.json?
{
    "data_dir": "data"
    "data_files":  {
        "train": "train-[0-9][0-9][0-9][0-9]-of-[0-9][0-9][0-9][0-9][0-9]*.*"
    }
}

we could also support lists for several configs

opened https://github.com/huggingface/datasets/issues/5694

I opened a PR to this PR to add data_files in YAML: https://github.com/polinaeterna/datasets/pull/1
builder_config:
  data_files:
  - split: train
    pattern: data/train-*

Let me open a PR to see if I can move the data files resolution outside of the MetadataConfigs to not modify it in-place

I wonder if we can make the cache backward compatible: we could just check if the cache directory with the old path exists. It will be useful for the research team which has a big datasets cache

I wonder if we can make the cache backward compatible: we could just check if the cache directory with the old path exists. It will be useful for the research team which has a big datasets cache

In the next PR maybe? :D
It's possible but requires some additional logic to correctly pass old config_kwargs (which used to include data_files but now it's None for builders from metadata) to generate the hash which is used to create the path.

If we only consider datasets that were pushed to hub, it's just a matter of using "{username}parquet" instead of "{username}{dataset_name}" in the cache directory name. The hashes stay the same :)

Did a small PR for backward compatibility, it was easy to add in the end: https://github.com/polinaeterna/datasets/pull/3

Just created a branch dev-3.0 in which we can merge this one and the other datasets 3.0 related PRs

@lhoestq why can't we merge it in main?

We can, it was just in case we had other things to merge after @mariosasko or @albertvillanova 's reviews

@lhoestq @albertvillanova @mariosasko we agreed on having configs (in plural) as a metadata field in readme but apparently Hub's yaml validation doesn't allow it to be not a list :D

(with config (in singular) it works)

we agreed on having configs (in plural) as a metadata field in readme but apparently Hub's yaml validation doesn't allow it to be not a list :D

If the configs field is specified in the YAML, the Hub can use it to improve the Use in dataset library snippet by listing the possible config values in load_dataset. So I think this needs to be fixed on the Hub side.
PS: I couldn't find an instance of someone using this field on the Hub, so I think using it for this feature is OK.

I couldn't find an instance of someone using this field on the Hub, so I think using it for this feature is OK.

@mariosasko I think it's because @lhoestq renamed configs to config_names in all canonical datasets :D so yes, configs field is now supposed to include custom configuration parameters introduced in this PR, and config_names is used (not really used lol) for list of strings of config names. It's being fixed on the Hub's side https://github.com/huggingface/moon-landing/pull/6490

There is one edge case I forgot to mention in the reviews - I think it's a good idea to support passing config params that are functions (Pandas uses them a lot) using this API (e.g. converters in the CSV config for converting a string column into a sequence). I see two solutions: string blocks with Python code in YAML or PyYAML tags.
But I think this can be addressed later.

I'm resolving the conflicts and writing some docs :) let's merge this soon !

authors: HuggingFaceDocBuilderDev lhoestq mariosasko polinaeterna


text:
High inference time when using chunk size 15

Hi @sanchit-gandhi !
I'm in the process of integrating multiple whisper backends into a unified package that includes VAD-based chunking. During testing, I observed significantly higher inference times while using the HuggingFace pipeline with distil-whisper. You can find the details here: https://github.com/shashikg/WhisperS2T/releases/tag/v1.1.0 [A30 GPU]
Could you please review the benchmarking script I'm using? It's available at: https://github.com/shashikg/WhisperS2T/blob/main/scripts/benchmark_huggingface_distil.py
Thanks for your assistance!
Shashi

Hey @shashikg! Thanks for sharing these benchmarks! I've had a look through the code, there were two variables that we could maybe adjust:

num_workers: is there any reason we pin this to 1 data loader num worker here? We could pre-process our data faster if we left this as the default (8)
chunk_length_s: worth setting this to 15 in all instances, e.g. here

Hey I think HF ChunkPipeline sets anything greater than num_worker>0 to num_worker=1. See here. Though I will once run the benchmark after setting this to a higher number.
That should not be an issue, for distil-whisper I only ran the benchmark on KINCAID WAV. this

authors: sanchit-gandhi shashikg

repo: huggingface/distil-whisper


text:
[Minor fix] Typo

traget -> target
cc @lvwerra

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint.

authors: HuggingFaceDocBuilderDev cakiki

repo: huggingface/evaluate


text:
fix: image for Deploy To Spaces svg

Updates the URLs to use raw instead of resolve which gets the svgs to show up properly.

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

hmm this is weird, what is the issue with resolve? i think resolve is the intended way to do this but cc @coyotte508 for guidance

Ah thanks for the explanation, I did indeed go to that url

repo: huggingface/hub-docs


text:
Streaming responses from scaled-to-zero Inference Endpoints return undefined

Hey folks, what's the recommended way to deal with the initializing period of a scaled-to-zero Inference Endpoint when using .textGenerationStream()?
For example, when using .textGeneration() directly, we get the 503 error directly that we can catch and maybe retry on.
However, when using .textGenerationStream(), fetching the next response chunk appears to succeed, except the .value at that chunk happens to be undefined. Reproducing code:
import { HfInferenceEndpoint } from "@huggingface/inference";
import { experimental_buildLlama2Prompt } from "ai/prompts";

const hf = new HfInferenceEndpoint(
  "DEPLOYED_LLAMA2_ENDPOINT",
  "API_TOKEN",
);

const response = hf.textGenerationStream({
  inputs: experimental_buildLlama2Prompt([
    {
      role: "user",
      content: "hello",
    },
  ]),
});

response.next().then((res) => {
  console.log("print next chunk");
  console.log(res.value);
  console.log("done");
});

Running via terminal results in the following:
> npx ts-node hf-test.ts
print next chunk
undefined
done

My (possibly naive) expectation would be for the generator returned by textGenerationStream() to wait for the corresponding Inference Endpoint to fully initialize -- maybe with exponential backoff -- before allow the chunk to be returned via .next(). But I am a novice here so I could be missing something. 😆
Is this expected behavior or a bug?

@co42 @Narsil , does HF endpoints support wait_for_model like Inference API?

It would be nice for API compatibiltity between inference API & inference endpoints!

@jinnovation you will now get a 503 in streaming requests with most recent version 2.6.5
With your current version, you can get a 503 if you set retry_on_error to false:
const response = hf.textGenerationStream({
  inputs: experimental_buildLlama2Prompt([
    {
      role: "user",
      content: "hello",
    },
  ]),
  {
    retry_on_error: false
  }
});

Soon the inference endpoint API will be updated, so that a call by default with @huggingface/inference will wait until the model is loaded (you can disable this behavior with retry_on_error: false to handle the 503 yourself)

Soon the inference endpoint backend will be updated, so that a call by default with @huggingface/inference will wait until the model is loaded

Fantastic! Thank you.


text:
Resource used to produce this version of dataset?

Can you provide any details about resources (CPUs, memory, storage, time) used to produce this dataset?
From the OLD/CC GitHub readme, I can estimate that, to get and process 20% of the August 2022 CC snapshot, which is 1.45 TB of data, requires about 15TB of storage disk and total deduplication will require about 700 to 900 GB of memory. But I can't find any details about how many CPUs were used and how much time it took to process it. Was this data processed on a single machine, single disk?

Yes, single machine and single disk. Particularly n2d-standard-224 running Ubuntu 20.04 LTS which you can find in the README.md. This machine has 224 CPUs and about 800GB of RAM. The disk that I used is about 15TB. The deduplication stage really pushes the RAM to its limit, so you need a machine with more RAM if you want to generate a dataset that is even a little larger.

Thank you for the information!

repo: huggingface/olm-datasets


text:
Add arbitrary scales config file

[SW-197077Add arbitrary scales config file

[SW-197077] arbitrary scales

Update run_generation.py

What does this PR do?

Fixes # (issue)
Before submitting

[ ] This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
[ ] Did you make sure to update the documentation with your changes?
[ ] Did you write any new necessary tests?

same as https://github.com/huggingface/optimum-habana/pull/1625

same as #1625

Yes, and I cherry-picked it into main

authors: libinta linoybu regisss

repo: huggingface/optimum-habana


text:
Add ONNX and ORT support for Falcon

This one was more painful than it should have been because:

Transformers without position_ids is bugged (repeat_interleave fails)
Transormers implementation reformats the KV cache directly in the model, which is new: https://github.com/huggingface/transformers/pull/26199
Optimum ORT has a rather bad support of MQA. Especially, the shape computation for IO Binding is very ugly with a lot of controlflow, expensive calls to normalized_config - this should be refactored with inheritance to avoid any controlflow at all.
ORT does not support bool input to Trilu op https://github.com/microsoft/onnxruntime/issues/16189

Remaining issue: I think repeat_interleave ONNX export inserts Loop in the ONNX, which we may want to avoid.

Fixes https://github.com/huggingface/optimum/issues/1172

repo: huggingface/optimum


text:
ValueError: When using DeepSpeed accelerate.prepare() requires you to pass at least one of training or evaluation dataloaders or alternatively set an integer value in train_micro_batch_size_per_gpu in the deepspeed config fileor assign integer value to Acc eleratorState().deepspeed_plugin.deepspeed_config['train_micro_batch_size_per_gpu'].

Hi, I have this error and I can't figure it out how to do a workaround. I want to run example...

Hi @reddiamond1234 , I met the same issue. Have you figured out how to solve it?

I have encountered with the same problem. How did you solve it?

@coder109 If you could provide a code example to reproduce your error, we can take a look and possibly help you. Also, please indicate the package versions you're using (the more up-to-date the packages, the more likely that some bugs have been fixed).

@BenjaminBossan Thank you for your reply! But I have solved my problem and the problem is caused by trl, not peft.

Great, thanks for the feedback @coder109. Is this something that was solved by upgrading trl to a newer version? It would be great if you could share in case other users encounter the same problem.

I think this problem, sometimes, is caused by some design flaws of some trl Trainers. Some trl trainers do not provide complete support for DeepSpeed.
If someone meets with this problem, you can refer to this issue of trl.

repo: huggingface/peft


text:
How to select a certain layer as token's representation?

My understanding from the paper that each token is represented by a 768-dim vector from the last hidden layer. Is it correct? If so, how can I get the second-to-the last layer's parameter as token representation?
model = BertForTokenClassification.from_pretrained("bert-base-uncased", num_labels=len(tag2idx))
list(model.named_parameters()) gives all weights but how can I know which parameters are from the second to the last layer?
for example, I have a word 'test', how can I get the BERT features from the second to the last hidden layer for the word 'test' in a given sentence?

Hi @yexing99,
What you need is the hidden_state, not the weightsof the model.
Don't use model.named_parameters() but just use the output of the model.
Here is an example: https://github.com/huggingface/pytorch-pretrained-BERT#bert
And more details here: https://github.com/huggingface/pytorch-pretrained-BERT#1-bertmodel
You should do somethings like this to get the BERT features from the second to the last hidden layer if the word test is for example the fifth word in your sentence (index 4 in the sentence):
encoded_layers, _ = model(tokens_tensor, segments_tensors)
features = [encoded_layer[4] for encoded_layer in encoded_layers[1:]]

repo: huggingface/pytorch-pretrained-BERT


text:
Add support for lightmaps

GLTF currently doesn't support lightmaps, but there is an extension being worked on: https://github.com/KhronosGroup/glTF/pull/1658
It would also make sense to include reflection probes and light probes in this.

Still WIP, will become suddenly important when we're trying to transfer a library like AI2THOR

repo: huggingface/simenv