text:
Improve small road highway tag guessing

For KLASS 8 and 9 these are usually better mapped to track in rural areas, and service in semi-urban areas. There's not enough information in NVDB tagging to now which is which though.
Maybe by looking at things like road density, distance to urban areas etc the script could make better guesses. Fully correct will not be possible, but the goal is to minimize the need for manual changes.

I tried to experiment further with combinations of attributes. I have not found a specific attribute which denotes that a highway is used for forestry, but "Tillgänglighetsklass" is supposed to be registered on those highways, according to instructions. If you select highways according to the following criteria, I think you get 90% of what you want for track:

Functional class 9 (class 8 are important highways in urban/mixed municipalities)
"Tillgänglighetsklass" > 2 (class 1 and 2 are highway quality roads)
No street name (excludes a few semi-urban highways)
Not already tagged as Unclassified

The rest to be tagged as Service.
This seems to work across rural, urban and mixed municipalities. It is not perfect. For example you get "islands" of service at the end of tracks, but those are easily discovered during import.

Tillgänglighetsklass seems rather quirky in rural areas, points out very strange roads as 1 and 2 in Lycksele for example.
Anyway, I'm also working on this, which includes not only analyzing tags but geometry etc. It will be a few more days probably depending on time I get to spend on it, but I'll report back and see what I got. The goal is to get as many as possible correct over the whole spectrum rural->urban, but noting that 100% is not possible, so not trying to do something overly complicated as it will probably not perform better than a simpler model.

I have now added (a rather large) patch to "guess" upgrade track to unclassified and service, plus fix a quirk where short unclassified stumps are upgraded to residential.
This patch has most effect on rural areas. On outer areas of Göteborg and other semi-urban areas the change to look at if gatunamn is used (instead of övrigt vägnamn) and make those service instead of tracks took care of most roads.
On rural areas however the KLASS 8 level is highly unreliable, as well as tillgänglighetsklass, so the patch looks at properties of the geometry and makes upgrades from there. Roads that form connecting links become unclassified, then short stumps grouped together are identified as likely driveways, and if those are connected to a larger road it's upgraded to service. In a second pass other likely driveways nearby are identified and upgraded, and then in a final pass short tracks that links driveways to a larger road is upgraded to service. Log output from a run on Lycksele below. The result is not 100% perfect, but reduces the manual work significantly. The algorithm could be further refined, but I think this is a good start.
2021-03-19 22:08:45,739 - process - INFO - Upgrading short unclassified stumps to residential (if connected to residential)...
2021-03-19 22:08:45,782 - process - INFO - done (46 segments upgraded from unclassified to residential)
2021-03-19 22:08:45,783 - process - INFO - Making best guesses of upgrading highway=track to highway=unclasified or service...
2021-03-19 22:08:46,015 - process - INFO -   5134 tracks to be considered for upgrading (skipped 325 as they had gates, 896 as typical track names)
2021-03-19 22:08:46,113 - process - INFO -   69 tracks upgraded to unclassified as they form connecting links
2021-03-19 22:08:47,078 - process - INFO -   391 tracks upgraded to service as likely driveways connected to larger roads
2021-03-19 22:08:47,156 - process - INFO -   69 tracks upgraded to service as likely driveways near others
2021-03-19 22:08:47,341 - process - INFO -   42 tracks upgraded to service they form short links between roads and driveways
2021-03-19 22:08:47,342 - process - INFO - done (1312 of 5134 tracks upgraded to service)

cool. is that already on the main branch?

Yes, during this development phase everything goes in on the main branch :-). It's probably ok to run a new batch run now, but there's been quite many changes and not a whole lot of testing so we may see failures...

running the batch now

Very interesting approach, and a better result. A bit refining is needed as there are still "islands" of unclassified and service.
A fair number of rural streets with homes are still getting highway=track. Would it make sense to first restrict highway=track to roads were the "Tillgänglighetsklass" attribute is present (disregarding the value, which is not consistent)? In that case only a few segments at the end of the road lacks any "Tillgänglighetsklass" and needs to be added to highway=track.
I looked mostly at Uppsala and Falun, which are mixed-use cases.

The problem with these type of algorithms is that there is just no information except the shape of the geometry, and if you tune it for one use case it will perform worse in another.
Note that I regard "track" wrongfully marked as "service" as worse than "service" wrongfully marked as "track", which leads to that I have excluded some aspects that will make service grasp over too much. I'll do a check on tillgänglighetsklass layer again.

Looked at Tillgänglighetsklass, it cannot be used as a exclusive cutoff, but it could be used in a weighting algorithm.
Tillgänglighetsklass is indeed missing on driveways, but also on shorter forestry roads and some longer. I think though that if looking if there are nearby "safe" driveways one can with quite good confidence upgrade shorter roads.
There are a few "impossible" cases, and that's single isolated dwellings with a single short (or longer) road, these are exactly the same as the many short forestry roads for roadside clearcuts. However, as soon as there are pairs closeby, one can quite safely assume it's driveways as it's very rare for forestry roads to make short stumps close together, as there's no need.

I ran a test on Åstorp which i small municipality in south of Sweden, quite good for making fast testing on mixed-use.
By converting tracks to service which lacks Tillgänglighetsklass but are less than 1000 meters long and have a service/residential road within 300 meters and going through all of the roads affected by this rule I got ~75% right, and ~25% wrong, and a few that should be unclassified or residential instead. Similar numbers in Lycksele, although I think it will be a higher number correct there. The problem is that the bad ones turn up in places you are less likely to look, ie away from urban areas in lesser mapped places. When working with places like Göteborg, Uppsala or Falun basically every road is already mapped in some shape or form. Often with not so good geometry and lacking in tags, but it's there and need manual merging.
That's why I think it's actually more important to get things right in less dense areas, than in more dense ones, and I think it's better to keep too many tracks than upgrade too many to service.
Not sure how much value it is spending a lot more time on this though, it takes a lot of time tuning and testing these algorithms and it's a delicate balance, I'll probably put this to the side for a while.
I have recently working with importing semi-urban areas, and it's a lot of work. I have started with a older version of the script so more highways are wrong type, but I can say it's not there most time is spent anyway. One thing is that there's a lot more driveways mapped in OSM than exist in NVDB, and all these need re-attaching. And there's a lot of alignment errors of buildings and farmlands and other natural areas that needs correcting to fit roads, and one cannot just bulk-move as the alignment error varies over the surface.

the batch completed and at least it looks like all osm files were generated correctly: https://nvdb-osm-map-data.s3.eu-north-1.amazonaws.com/index.html

I just checked for myself the area where I live and I think there is a problem that I just found:
https://www.openstreetmap.org/#map=19/59.36117/17.96452 the road "landsvägen" is supposed to be a "lokalgata stor" which I think should translate to a tertiary road? it's currently marked as "residential" and that's definitely not true :)

It's a tradeoff, making lokalgata stor to tertiary in big cities makes them really overflow with tertiary. While that might be considered okay if we would start from scrach, it's not how they are mapped currently. Below is the current translation rules for city roads (those that exist in the gatutyp layer).
As you see only "Huvudgata" is translated to tertiary, and only if Funktionell vägklass is 5. For other roads also vägklass 6 becomes tertiary.
I've tuned these translations a bit differently, but anyway you do it there will be quite much mismatch from how it's mapped in OSM currently. When there is a difference, I change to what's in OSM while merging, unless it's obviously wrong. When discussing this a while back on the Facebook group it was very clear that the traditionalists is not keen at all of the idea of using fixed translations from NVDB classes to primary/secondary/tertiary etc, but they want this to be a mapper judgement, so I don't think we can make a translation that all will accept. So I think we will have to live with having to adjust these types quite often.
        if gatutyp == "Övergripande länk":
            raise RuntimeError() # should already be handled
        if gatutyp == "Huvudgata":
            if klass <= 1:
                tags["highway"] = "trunk" # 0, 1
            elif klass <= 2:
                tags["highway"] = "primary" # 2
            elif klass <= 4:
                tags["highway"] = "secondary" # 3, 4
            elif klass <= 5:
                tags["highway"] = "tertiary" # 5
            else:
                tags["highway"] = "residential"
        elif gatutyp == "Lokalgata stor":
            tags["highway"] = "residential"
        elif gatutyp == "Lokalgata liten":
            tags["highway"] = "residential"
        elif gatutyp == "Kvartersväg":
            tags["highway"] = "service"
        elif gatutyp == "Parkeringsområdesväg":
            tags["highway"] = "service"
            tags["service"] = "parking_aisle"
        elif gatutyp == "Infartsväg/Utfartsväg":
            tags["highway"] = "service"
        elif gatutyp == "Leveransväg":
            tags["highway"] = "unclassified"
        elif gatutyp == "Småväg":
            tags["highway"] = "unclassified"
        else:
            raise RuntimeError("Unknown gatutyp %s" % gatutyp)

I understand the frustration :)  I made a few examples with Tilgänglighetsklass. It seems to be a sufficient universe from which to get track. Of course the service "islands" at the end of branches would have to be included. And I am sure your more sophisticated method could identify a few unclassified and service roads among the tracks.
I just think there is too much guessing going on now. It would be clearer to base tagging on what is actually contained in NVDB (for good and for worse, just like primary/secondary etc), and I have found no other attribute than Tilgänglighetsklass to denote forestry roads. Sure it will not be perfect. That is why we do a manual import.
My experience is that importing is equally useful in semi-urban areas because the existing mapping is often with a big offset and with missing maxspeed, street name etc. Downtown areas in cities, on the other hand, are often well mapped because more mappers live there.
You could of course also have a special tagging variant for import needs in the north of Sweden.

I agree that it's useful to map already mapped areas, but what you get there is preset highway types.
No-one should ever delete existing roads and replace them with imported. It's just plain wrong if we suggest that it would be a valid workflow, and if anyone is caught doing that they should not be allowed to do imports. The old should be modified to new, either manually or using replace geometry tool. So why is modify instead of delete relevant in this context? When you modify you get clear feedback to set your highway type to either keep the old or change to the script's best guess.
This is not just about service vs track. It's about service, track, residential and unclassified, in semi-urban places all these four types may have same type of NVDB tags. You can't just look at the script output and decide if you like that or not, you must also look at what is pre-existing in OSM, and make a fair judgement of what's the best call, not just pick script result because that's the easiest. When doing that, it's clear that semi-urban places require careful work.
That said I think that there's quite of lot of bad highway choices in OSM already, so I correct it a lot too.
One can try that discussion rely on a static translations with the traditionalists :-). They couldn't care less about what is in NVDB. If NVDB provides wrong information, they are not happy to accept that just because it's in NVDB. They just look at what we want to be in OSM, not looking at NVDB at all. And I agree with them. We should not change our ways to map just to match what can be done with NVDB. We should look at how these ways should be tagged in OSM and Swedish tradition, and only then see if NVDB can match that. It can't.
The purpose of a guessing algorithm is to minimize the need to manually change. If the guessing algorithm causes a larger need to change than using statically incorrect conversions from various unreliable NVDB tags, then I agree it's probably better using a static conversion.
I think a problem here is though that we don't agree about the degree of correctness of a static conversion, I think you believe it's more correct than I do.
Let's look at an example from Uppsala.
This is currently in OSM today:

This is static conversion output from your script (don't know if I have the latest):

This is the guesswork conversion from my script from Matthias latest batch run:

I want to make changes to all these, also what's pre-existing in OSM.
Let's look at Örkälla, southwest of Skyttorp. It's residential in OSM, service in your script and track in mine. Residential is best call, but there's no tagging in NVDB for that, that's why our scripts get stuck on track/service.
Look at the forestry roads to the east. Here I say that my script has even better output than what's currently in OSM, which has too many unclassified. You don't want have gated roads as unclassified unless you have local knowledge and knows that the gate is permanently open or there's some other special reason. When it comes to quarries these are often abandoned, and type of road depends on if it's in active use or not.
Again, track is not tractor track exclusively in Swedish mapping tradition, it's also less important unpaved roads, which we have a lot of.
On secondary vs tertiary my script matches OSM, and the reason for that is that I use FPV-klass rather than funktionell vägklass for numbered roads which better match what OSM has now (but still not always...)
On the service roads, my script has made few mis-upgrades, but obviously misses quite many, and the reason of that is because it becomes too hard to differ between what's most likely roads to houses and what's forestry/argiculture tracks. However, these are generally in clusters which fit in your edit view, and I think it's better to have issues there than on long-winding roads.
For this example I certainly prefer working with the output the guesswork output makes rather than static rules.
Below a zoomed in example from Gödåker, which is seen center left on the others, after I have manually pre-processed the highway types:

Some short ones needed upgrading to service, but there's also some split work, Fredrikslund starts as service and transitions into track, ending with a turning circle (which needs adding, no turning circles on small roads in NVDB)
Gödåker and Lilla Gödåker, service, unclassified or residential? I have made it unclassified as there are several houses reached and the road is not specifically to one house. One might argue this should be residential, and I'm not entirely sure on this, but my current strategy which seems to fit mapping tradition well is to use unclassified unless there's a certain level of density when it becomes residential. Likewise Husby is split between unclassified and service.

Haha. Amazing thanks for the input. I just picked something close to me at random and sure enough found the hardest problem ;)
I really enjoy all the in-depth explanations. thanks a lot!

It seems you have not looked at my examples (which by the way only was produced to show the extent of tracks, not a complete end product). And I also seem to have failed to explain my approach. But I give up now.

Oops, I did miss the examples link, sorry for that, looking into it!

Here's a screenshot from the Tillgänglighetsklass script run, again sorry for missing that:

On a related subject: what I think we will have to do if we want to formalize this in an import process "available to all on full scale" is to start in a different end: making a document describing how highway types should be mapped in Swedish conditions, and specifically focusing on residential/unclassified/service/track, and get that accepted by the Swedish OSM community.
This document should not mention NVDB at all. It's just OSM on OSM's own conditions. In theory the OSM wiki should already have all the answers, but Swedish conditions are a bit special and there's an existing mapping tradition to take into account etc, and there are lots of corner cases.
I've learnt that examples is best, the document would contain similar to the screenshots above with different type of conditions and which types to use when.
When we have that established, the next step is to see how well this maps to what NVDB can provide. With that document in place, we have a common reference and it's easier to argue for what's correct in an OSM context.
Then one can start working on tuning the script to make as good job as possible to match that document guidelines, and then make a import process which describes caveats where one need manual changes etc.
I think it's the wrong way to look at what NVDB can do, and then convince ourselves that it is the best way to use OSM highway types. Even if NVDB functional class data would be 100% correct and consistent, which it isn't especially in the 7-9 range, highway types is not really defined in the same way as NVDB uses functional classes.
Sure for the range trunk to tertiary, one could argue for a static conversion, because there functional class maps to the highway types pretty 1 to 1, it's just that OSM mappers have in many places made a bit different decisions than NVDB, and NVDB itself has slightly different functional classes in different layers on these roads (depending on traffic type etc), so even there I think that manual adjustments are to be expected.
However, when it comes to residential, unclassified, service and track it becomes more complicated, it's sort of both functional class and use based. NVDB has no information if a road is used as a driveway. It indeed has information about residential, but much more area-based to maxspeed=50 areas and it misses quite a lot of villages that traditionally get residential in OSM. Actually I think I may be under-using residential myself currently, it's one thing I'll take up with the community. Service is about accessing something. And then we have track which is used a bit special in Swedish conditions, as we have this huge forestry network.
Making such a document will be lot of work though, and many discussions of people of differing opinions. I'm not totally against the idea of making such a document myself at some point, but it's probably not going to happen this year.
For my own purpose, the script is mature enough to entertain me with import work for years. As long as I am the single user that uses it on scale, I have lots of eyes on me, and informal explanations and discussions of what I'm doing is enough.

Here's a test with both tillgänglighetsklass and guesswork track upgrade active:
www.torger.org/anders/downloads/uppsala-new.osm
Note that I haven't run it with all layers so there's info missing, but most related to highway tag is there.

So I have added --small_road_resolve where you can specify prefer_service, prefer_track, or its static variants. Without static, it will on top run the guesswork upgrading of remaining tracks to unclassified/service (ie after static rules have been applied)
This is "prefer_service_static", should be most similar to NKAs, download here: prefer_service_static

This is "prefer_service", ie some track upgrades applied, not big difference in this view, but one can see some tracks have become unclassified,
prefer_service

And finally this is "prefer_track", ie same as the latest batch run
prefer_track

Just let me know which one you prefer as default setting, or if you want something else.
In the end I'm 100% sure that we will not get acceptance to use NVDB-derived highway tags, guessed or static, straight off as-is. Manual inspection and adjustment will be an absolute requirement on the importer. Just as using modify instead of delete followed by insert is an absolute requirement.
And then the task for the script is just about making an output that the importer finds effective to work with, and there we will see different preferences I suppose.
My preference is as follows: for roads that need tag adjustment I prefer when they are clustered (for example too many tracks inside a village is easier to correct than too many service spread out in the forestry network), and if I do mistakes I prefer to leave one too many with track tag than one too many service or unclassified tag. The reason for that is I think it's worse indicating that a road leads to something (service) when it doesn't than the other way around. On track vs unclassified I think when mistakes occur it's better with under-classing than over-classing.
By having a few different rendering options maybe we can cover most preferences.
What should the monthly batch run use though? Maybe two of the most popular. One reason to have two is to make it really clear for those that don't read docs that highway tag isn't set in stone.

I wonder if it would be an option to leave the highway tag out completely for the most "disputed" tags? That way they have to be set manually? Maybe that's too much work when actually importing the file...
Another option would be to set the flag by "kommun". So e.g. we know that setting X works best in Stockholm but setting Y works best in rural areas.
By the way, I have tinkered a bit further and now created a pipeline that produces vector tiles.
Have a look here: https://nvdb-osm-map-data.s3.eu-north-1.amazonaws.com/map.html#7.8/64.489/18.735 Currently on this map you see the differences in the tag highway. Additionally, when you zoom in a bit further, you get an overlay of the geometry that the nvdb2osm script outputs with the highway tag on the line. Maybe that helps.
My plan is to have this map toggleable so you can toggle if you want to see where the most differences are in lanes, maxspeed, highway etc.
But maybe I'll leave the highway tag out completely if it does maybe more harm than good.

I've also thought about leaving out the highway tag from script output. When working with previously mapped areas it's in theory quite practical actually, as you then never need to resolve highway tag conflict, with replace geometry you will automatically use what was previously there. However, it becomes a little bit messier to work with visually in the editor as the geometry then just becomes gray lines. It's also extremely easy for the user to remove highway tag from the layer if he or she wants, just select and delete.
If we are afraid that people are going to use the pre-made batch run data carelessly one could indeed leave it out of those files just for that reason. However, I'm not too worried about that, I think it's more about how we lay out the documentation, most want to do a good job. If one is worried about careless highway tag override I would be much more worried about people just deleting everything in view and pasting in the new layer, or messing up route relations, or turn:lanes on the bigger roads etc. There's so many wrongs you can do in addition to highway tag which we can't do anything about, except policing.
In terms of visualization of your comparison map (really nice by the way, just tested the new version), I'm thinking that maybe just differing between cycleway/footway/path and then all other highways is enough, and perhaps not with text which tend to distract overviews, but instead with different line width or dashed vs solid or something. When scanning for differences to find something suitable place to fix one will not look for mismatching highway tags, but things like maxspeed and geometry offsets.

Yeah you're right, maybe we're overly worrying...
Good idea. I'll put more effort down into the strengths of NVDB which is geometry and speeds, lanes etc and build the diff map around that.
I guess it would be really nice if you and @NKAmapper could come to a conclusion around the tagging so that we then can prepare the proposal for the import (for which NKA has already made a nice wiki page)...
My primary intention is really just to help as many people to improve OSM rather quickly :)

With risk of sounding a bit negative again, I think it will be tough to get a lot of people to actually help do the import work at least on short notice, so I would not feel too much in a hurry of the import proposal. This script has been out for two months, I've talked about it and the way I have put it to use frequently on the OSM Sweden facebook group, and still it's only the three of us that's active here on github. I was a bit naive and thought that there would be a bit more interest. There's no lack of appreciation of the import work I've already done, it's not that, but it's quite clear that among the active mappers there's no hunger for using import scripts to make their job quicker.
Part of the reason is that many mappers feel that it's a bit too difficult and they are afraid of making mistakes. I'd say that it's actually not that difficult, but if you never have used JOSM before there's a bit of a threshold for sure.
I've thought about making a "youtube training video" with JOSM and the import script and merging techniques. Text documentation gets a bit dense, the readme file to this script is already pretty crazy. I think a video could be very good way to get new mappers take the step and join the work. I've done some lame but effective instructional videos before for other reasons, so it's something I could do, but probably not until after summer if so.
There are a handful of advanced mappers in the active Swedish community that have the know-how already of course. I know of a few that already use NVDB since quite a while, I got my initial pointers to making my script from them, but they have their own scripts and methods, and still after knowing about my considerably more advanced and detailed script (ie much richer tagging) they still use their own methods and haven't even tried this script or commented. The thing is that I think fewer than we think has a specific desire to make OSM a complete working end product for whole of Sweden, but rather has pretty local interests or just find like mapping as a relaxing activity where the speed of mapping or richness of tags is not really that interesting.
So even with a fancy import guide, if we don't do anything more than that to attract mappers I think there's a big risk that there will only me and NKA doing all the full-coverage importing... to me that wouldn't be a failure though, I'm happy that I now can map much faster than I did before, I enjoy the mapping activity more this way, but sure I can't realistically finish Sweden on my own. In time I can cover my local interests though.

FYI: I collect edit guidelines based on what I use myself and on discussions in the facebook group in the README file. This should probably be refined with image examples and moved to an appropriate guideline document on the OSM wiki and be peer reviewed at some point. I've just kept it there for now for my own convenience. Anyhow, under the anonymous heading "Övrigt att tänka på" there's guidelines of how to relate to the script's highway tags and other things.

@atorger something completely different: I just found this: https://nvdb-osm-map-data.s3.eu-north-1.amazonaws.com/map.html#12.08/59.96192/15.85696 Is your script running amok? 😆 seems like it generates quite some chaos...

On the topic of your longer message: I do understand that it will probably not be a huge group that does the import and that's fine. Maybe a youtube tutorial is actually a really good idea. Part of me programming the pipeline and the map is the hope that this would lower the barrier of entry for some people. (OK and because it's actually really fun to tinker with these things 😉)
But as I understand the OSM customs, we should have an import plan ready and OK-ed by the community, right?
I would like to help with this as much as possible but this is a completely new field to me. I'd like to have @NKAmapper's guide for this as much as possible as you have already successfully driven such projects.

@matthiasfeist that's Fagersta kommun which looks ok in JOSM if you download it from the batch run page. If something like that would happen in the script it would never complete, so there's something going on somewhere else in the pipeline.
Yes I think your map will not only lower the barrier, but also make more people interested.
Yes we should have an import plan, especially if more people starts getting involved. I realize that it may be a bit problematic or at least uncomfortable to publish your "lower barrier of entry" project without having a import plan in place. I guess I could help with some text, just point me where I should write :-).
I don't know how you roll these things, but I guess one can invite people from the international mapping community to help, afterall many of the most active mappers in Sweden are not Swedes, top list here: https://osmstats.neis-one.org/?item=countries&country=Sweden. If we do that I think it becomes even more important with a solid import plan.

:) yes it seems to be a bug in my script. Damn...

Closing this. If we need further refinements, new algorithms etc, open a new issue.

authors: NKAmapper atorger matthiasfeist

license: Unlicense


text:
Error on start up of secondary without config.yaml

When starting the secondary from the command line without the config/config.yaml file in place I get the error below.. I believe the default value for notificationQuarantineDuration should be put in place if the config.yaml is not found like the other values.
This could I think prevent the secondary starting up in some enviroments..
With no config.yaml in place
secondary_1  |
secondary_1  | FINER|2021-04-16 01:29:17.092662|CommitLogKeyStore|last committed sequence: 1
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:29:17.100245|HivePersistenceManager|scheduleKeyExpireTask starting cron job.
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:29:17.101842|HiveKeystore|hive key:privatekey:at_secret
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:29:17.103237|HiveKeystore|hive value:AtData{data: 1234567890qwertyuio, metaData: {createdBy: null, updatedBy: null, createdAt: 2021-04-16 01:29:17.101832Z, updatedAt: 2021-04-16 01:29:17.101832Z, availableAt: null, expiresAt: null, refreshAt: null, status: active, version: 0, ttl: null, ttb: null, ttr: null, ccd: null, isBinary: null, isEncrypted: null, dataSignature: null}
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:29:17.105754|AtRefreshJob|scheduleKeyRefreshTask runs at 6 hours
secondary_1  |
secondary_1  | Unhandled exception:
secondary_1  | NoSuchMethodError: The method '[]' was called on null.
secondary_1  | Receiver: null
secondary_1  | Tried calling: []()
secondary_1  | #0      AtSecondaryConfig.notificationQuarantineDuration (package:at_secondary/src/server/at_secondary_config.dart:544)
secondary_1  | #1      new ResourceManager._internal (package:at_secondary/src/notification/resource_manager.dart:25)
secondary_1  | #2      ResourceManager._singleton (package:at_secondary/src/notification/resource_manager.dart:13)
secondary_1  | #3      new ResourceManager.getInstance (package:at_secondary/src/notification/resource_manager.dart)
secondary_1  | #4      AtSecondaryServerImpl.start (package:at_secondary/src/server/at_secondary_impl.dart:173)
secondary_1  | <asynchronous suspension>
secondary_1  | #5      SecondaryServerBootStrapper.run (package:at_secondary/src/server/bootstrapper.dart:50)
secondary_1  | <asynchronous suspension>
secondary_1  | #6      main (file:///buddy/atsign-open-source/at_secondary/at_secondary_server/bin/main.dart:21)
secondary_1  | <asynchronous suspension>

But putting the file in place the secondary starts up just fine.
secondary_1  |
secondary_1  | FINER|2021-04-16 01:31:17.780268|CommitLogKeyStore|last committed sequence: 1
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:31:17.787286|HivePersistenceManager|scheduleKeyExpireTask starting cron job.
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:31:17.787542|HiveKeystore|hive key:privatekey:at_secret
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:31:17.787573|HiveKeystore|hive value:AtData{data: 1234567890qwertyuio, metaData: {createdBy: null, updatedBy: null, createdAt: 2021-04-16 01:31:17.787533Z, updatedAt: 2021-04-16 01:31:17.787533Z, availableAt: null, expiresAt: null, refreshAt: null, status: active, version: 0, ttl: null, ttb: null, ttr: null, ccd: null, isBinary: null, isEncrypted: null, dataSignature: null}
secondary_1  |
secondary_1  | FINEST|2021-04-16 01:31:17.788265|AtRefreshJob|scheduleKeyRefreshTask runs at 14 hours
secondary_1  |

On further looking the docker container includes the /atsign/config/config.yaml so that saves the day. But without the config file in the container (I have overlaid the /atsign directory) by using a volume the defaults values should still be in place if memory serves.

source: gharchive/issue

license_type: permissive

license_source: github-api


text:
ci: Split e2e tests

- What I did
Major change:

Separated the e2e tests into 3 groups, and simplified them where possible.

Group 1 - local against trunk - runs pretty much always
Group 2 - local against installer - runs pretty much always, since the installer's prod is trunk branch
Group 3 - local against old-releases - runs only on a manual workflow call

These changes were done to reduce the number of times we trigger backwards tests, but also to simplify things so that @JeremyTubongbanua can implement tests against prod rvds more easily. (He will implement them similar to how Group 3 is implemented)
In addition, I made the following minor changes requested by @cpswan :

[x] README for act wrapper
[x] Filename more explicit (i.e. act run)
[x] Note where some of the ../ paths are pointing to
[x] Comment why shellchecks are disabled

- How I did it
- How to verify it
- Description for the changelog
ci: Split e2e tests

As you can see, when the PR is created, the backward (release) tests are skipped. Before reviewing this, I'd like to manually run against this branch to confirm that those tests are working as intended. I will link that workflow run to this PR.

I broke the atKey loading from secrets, by creating and trying to use the env var in the same step... confirming locally before I push the fix

Here are the additional backward tests which I've triggered: https://github.com/atsign-foundation/sshnoports/actions/runs/6034676624

source: gharchive/pull-request

repo: atsign-foundation/sshnoports


text:
improve performance of 'noms diff' - parallel at leaves

Even after landing #1961 we have an opportunity to improve diff. The latency of a remote data store is causing diff to continue to be slower than necessary. We can do more work to parallelize finding diffs for the leaf sequences.  Should help all collection types.

Our problem is now too much parallelism at the leaves

authors: mikegray rafael-atticlabs

repo: attic-labs/noms


text:
Go: List.Map should return List

Currently List.Map returns []interface{}. It would be better if it returned a Noms List.

I can see valid reasons for wanting the resulting collection to be chunked and not. Are you thinking about specific use cases?

Seems like https://github.com/attic-labs/noms/issues/2559 encompasses this now

authors: arv kalman rafael-atticlabs


text:
swift matmul: avoid unnecessary allocations

Those extra allocations slow down the code in and of themselves and later contribute to more array CoW checks during mutation.
In my tests, this gets the duration from over 9 seconds down to 0.6-0.8 seconds.

Thank you! Figure updated. Now swift is faster on nqueen+matmul but sudoku stands out. Is it also caused by 2d vectors?

I wrote another PR (#31) to use a similar technique in the sudoku code. It does bring the time down to 9-10 seconds, which is still a lot, but I don’t know how to optimise it any further. The profiler shows lots of retain/release calls taking most of the time, which is pretty typical for Swift benchmarks.

I’ve also simplified the subscripts here. I didn't realise at first that the parens aren’t necessary: #32.

I've figured out where the retain/release calls come from and fixed that: #36.

authors: attractivechaos yakubin


text:
Print not working on bluetooth printer

The open cash drawer works but it doesn't print on TSP100 bluetooth printer. It doesn't even throw any errors. I haven't tried other printers as that's the only one I have. Any suggestions? Thanks.

Hi! I was wondering if you have already tested this plugin in TSP100 printer? Thank you! 😊
Best regards,
Esthon Wood

On 20 Jan 2018, at 5:40 AM, auctifera-josed notifications@github.com wrote:
Hi @esthonwood https://github.com/esthonwood I thought I had a TSP100 in the storage room, turns out they sent us another TSP700II by mistake, it'll take a while to change it so I can test.
But I went over the SDK Documentation http://www.starmicronics.com/support/ZipFile.aspx?sat2=203&id=203&type=4&referrer=http://www.starmicronics.com/support/SDKDocumentation.aspx&tabText and it looks like the TSP100 uses a different emulation (StarGraphic) than TSP700II (StarLine) and I only use the StarIoExtEmulationStarLine all over the code so that's why you are getting this alert from line 606 because in line 604 the _printerManager.port will always be null.
I believe just changing the emulation (or have it dynamically assigned depending on the printer) will make this plugin work with the TSP100 emulation.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/auctifera-josed/starprnt/issues/3#issuecomment-359096833, or mute the thread https://github.com/notifications/unsubscribe-auth/ADP3_yHiveAOsS9PgAV-KQfqbyoYeeRFks5tMQvbgaJpZM4RaOhV.

Hi @esthonwood, sorry man I've been really busy and haven't been able to work on this, I believe all you need to do is change the StarIoExtEmulationStarLine for StarIoExtEmulationStarGraphic which is the TSP100 emulation.

Hi! I changed the emulation to StarIoExtEmulationStarGraphic but it didn’t print anything. Can you at least tell me if my code is correct? Because I keep getting errors like "Please connect to the printer before sending commands." and "Please re connect to the printer, something is not working as expected."
I desperately need this to work on TSP100 printer. I really appreciate you taking the time to help me. Here’s a snippet of my code:
declare let starprnt: any;
export class TestPage {
starPort: any;
openCashDrawer() {
	starprnt.connect("BT:TSP100", function(err,res) {
		if (err) {
			console.log('FAILED: ' + JSON.stringify(err));
		} else if(res) {
			console.log('OK: ' + JSON.stringify(res));
			var openOK = function(res){
				console.log('OK: Did it actually open? Data: ' + JSON.stringify(res));
			}

			var openErr = function(res){
				console.log('FAILED: ' + JSON.stringify(res));
			};
			starprnt.openCashDrawer(openOK, openErr);
		}
	});
}

printReceipt() {
	starprnt.connect("BT:TSP100", function(err,res) {
		if (err) {
			console.log('FAILED: ' + JSON.stringify(err));
		} else if(res) {
			console.log('OK: ' + JSON.stringify(res));
			var printOK = function(res){
				console.log('OK: Did it actually open? Data: ' + JSON.stringify(res));
			}

			var printErr = function(res){
				console.log('FAILED: ' + JSON.stringify(res));
			};

			let content = "Star Clothing Boutique\n123 Star Road\nCity, State 12345\n";

			starprnt.printData(content, printOK, printErr);
		}
	});
}

}
Best regards,
Esthon Wood
esthonwood.trulyrichclub.com

On 26 Feb 2018, at 10:20 PM, auctifera-josed notifications@github.com wrote:
Hi @esthonwood https://github.com/esthonwood, sorry man I've been really busy and haven't been able to work on this, I believe all you need to do is change the StarIoExtEmulationStarLine for StarIoExtEmulationStarGraphic which is the TSP100 emulation.
—
You are receiving this because you were mentioned.
Reply to this email directly, view it on GitHub https://github.com/auctifera-josed/starprnt/issues/3#issuecomment-368518075, or mute the thread https://github.com/notifications/unsubscribe-auth/ADP3_4kcfKl3hH5f99TrjYm8r-wKd3xvks5tYr3IgaJpZM4RaOhV.

Hi @auctifera-josed! I changed the emulation to StarIoExtEmulationStarGraphic but it didn’t print anything. Can you at least tell me if my code is correct? Because I keep getting errors like
"Please connect to the printer before sending commands." and
"Please re connect to the printer, something is not working as expected."
I desperately need this to work on TSP100 printer. I really appreciate you taking the time to help me. Here’s a snippet of my code:
declare let starprnt: any;

export class TestPage {
	starPort: any;

	openCashDrawer() {
		starprnt.connect("BT:TSP100", function(err,res) {
			if (err) {
				console.log('FAILED: ' + JSON.stringify(err));
			} else if(res) {
				console.log('OK: ' + JSON.stringify(res));
				var openOK = function(res){
					console.log('OK: Did it actually open? Data: ' + JSON.stringify(res));
				}

				var openErr = function(res){
					console.log('FAILED: ' + JSON.stringify(res));
				};
				starprnt.openCashDrawer(openOK, openErr);
			}
		});
	}

	printReceipt() {
		starprnt.connect("BT:TSP100", function(err,res) {
			if (err) {
				console.log('FAILED: ' + JSON.stringify(err));
			} else if(res) {
				console.log('OK: ' + JSON.stringify(res));
				var printOK = function(res){
					console.log('OK: Did it actually open? Data: ' + JSON.stringify(res));
				}

				var printErr = function(res){
					console.log('FAILED: ' + JSON.stringify(res));
				};

				let content = "Star Clothing Boutique\n123 Star Road\nCity, State 12345\n";

				starprnt.printData(content, printOK, printErr);
			}
		});
	}
}

Hi  @esthonwood  I have added emulation and the portsettings to the check status function on iOS, use the plugin from my fork and let me know if it works on the TSP100. you can see some examples from my test app. I am working on updating all the functions to support both printing methods SMPort(pass port and emulation) and StarioExtManager (send commands to connected printer)

Hi @infoxicator, did you also update the StarPRNT SDK version to 5.5.0?

Hi @esthonwood I have updated it on my fork, but I am testing it at the moment I will commit the new version soon... Did you manage to connect to the TSP100?

Hi @infoxicator it still won't print on the TSP100. Please let me know when your fork is ready to use. Thanks.

Can reproduce, tried doing this in the js console
window.starprnt.printRawText(
  'BT:TSP100',
  'StarGraphic',
  { text: 'lfkajsldfjajkdljfkljklfこんにちは' },
  s => alert(JSON.stringify(s)),
  e => alert(JSON.stringify(e))
)

I get the success alert but nothing prints. Using .print() I can only get the appendCutPaper print command to elicit a response from the printer, no other print commands seem to do anything.

@esthonwood I emailed star micronics sales support in Japan, and they said that TSP100iiiBI (starGraphics emulation) only supports printing raster images.

authors: auctifera-josed daniellizik esthonwood infoxicator

repo: auctifera-josed/starprnt

license: mit

license_source: bigquery


text:
Tooling Setup

Set up TypeScript in the project.
Set up code linters (ESLint) and formatters (Prettier).
Configure compiler/transpiler and bundler if needed (e.g., Babel, Webpack).

This Pull Request includes the foundational setup for our development environment. The following tasks have been successfully completed:
TypeScript Setup: The project has been configured to use TypeScript, ensuring we leverage the benefits of strong typing throughout our development process.
Code Linters and Formatters: ESLint and Prettier have been set up. This ensures that our code is consistent and follows the best practices as defined by our linting and formatting rules.
Compiler/Transpiler and Bundler Configuration: The initial setup for code compilation and bundling is in place. We are utilizing TypeScript's default capabilities for compiling, and Webpack has been configured for more complex bundling use cases, such as Hot Module Replacement (HMR).

repo: augustocfreitas/ChronoConquest

license: MIT


text:
Option to show transactions only in the account, but not children [feature-request]

beancount-web currently shows:
parent-only total: dark black
parent + children total: gray
This is very useful. However, when one clicks on the account, there is no way to see a list of the transactions for only the parent account, without also seeing the the children's transactions intermingled. It would be great to add an option to the set of options (open, close, transaction, etc.) to turn off display of transactions from children accounts, so one can see what transactions add up to the dark black number mentioned above.

Nice, thanks!
When I set journal-show-childentries = False, The Treemap and bar chart still seem to include children, while the line chart doesn't, if I'm seeing this right?

That is correct, because if you set journal-show-childentries = False the treemap would only show one rectangle - the account you are viewing. Does that seem wrong/unintuitive to you?

Correct, that does seem unintuitive to me. In certain cases-  in particular, when a leaf account is chosen - the treemap simply serves as a visual confirmation of the selected context rather than providing ratio information. It would be confusing if the tree map show anything else but a solid rectangular block in this case.
Choosing a parent and excluding the leaves is the same situation, IMHO, and it is more important to keep the table and visual representations in sync than provide ratio data.
In my personal experience, I started doubting what the table was showing and had to go through all the transactions to determine whether or not child accounts were being shown even though I had asked for them to not be shown.

authors: aumayr redstreet

repo: aumayr/beancount-web


text:
Add information for webpack users (uglifyJs plugin)

Hi,
as I've learned it the hard way, you might share this information on your wiki. When using aurelia-materialize-bridge (and thus materialize) via webpack with the UglifyJS-plugin (standard in aurelia webpack skeleton production build) you have to make sure to except the materialize module (via exclude: ['materialize'] if your module is named materialize) from the UglifyJS plugin. Otherwise the names will be mangled and the materialize module won't load correctly (missing dropdown). Another option would be to disable mangle as a global setting on the UglifyJS settings object.
Thanks
Joscha

Thanks! I will add this.

authors: JoschaMetze Thanood


text:
webpack json loader?

Hi,
I'm trying to add JSON loader support to the skeleton-esnext-webpack project. I've added:
require("@easy-webpack/config-global-json")()
to each case in the advanced configuration in webpack.config.js
but I'm getting the following when trying to import cfg from ./package.json into the project:
ERROR in ./package.json Module parse failed: C:\Users\Miles\skeleton-esnext-webpack\package.json Unexpected token (2:8) You may need an appropriate loader to handle this file type.

General questions are best handled through our Gitter channel (gitter.im/aurelia/discuss) or Stack Overflow. @niieani may be able to assist.

@mhingston there is no @easy-webpack/config-global-json module. Try: @easy-webpack/config-json.

authors: EisenbergEffect mhingston niieani

repo: aurelia/skeleton-navigation


text:
React to slot content changes in a custom element

I'm trying to develop a custom element that needs reacting to changes to inner html, so basically one can write:
<nx-jstree live-html.bind="true">

${node.text}

</nx-jstree>

The custom element basically has this template:
<template>

<slot></slot>

</template>

and I tried to add a MutationObserver on the src div, but it isn't capturing any change..
If it's supported, what's the official way to get notified when the slot content changes?

I want to add that I tried to use a @child attribute to reference the second div (the one that contains the slot), but it is always null. It seems that child and children selectors don't work at all.

@nexbit Are you expecting to capture list mutations?

@levan-g Yes, but more generally I want to be able to react to any change in the slot content.
The user of my component should be able to use whatever technique to assemble the UL...
With a custom attribute I don't have any problem observing changes with mutation observers, but the new slot custom element seems to not support observers.
With the latest RC even the child and children selector do not work reliably as before, and it's not stated anywhere if they are meant to work with `slot

So I created a custom attribute for you.
@inject(Element)
@customAttribute('mutation-observer')
export class MutationObserverAttribute {

	constructor(element: Element) {
		this.element = element;
	}

	attached() {
		this.observer = new MutationObserver((mutations) => {
			this.value(mutations);
		});
		this.observer.observe(this.element, {
			characterData: true,
			childList: true,
			subtree: true
		});
	}

	detached() {
		if(this.observer) {
			this.observer.disconnect();
			this.observer = null;
		}
	}
}

usage
<template>

<slot></slot>

</template>

It seems like there may be a bug in child/children, since those should work. They work in our tests. We will need to try to reproduce the issue to see what is going on.

@EisenbergEffect for me @child and @children works correctly

I'm closing this because I think that child/children are being used incorrectly. They select the content of the element, not elements inside it's view. Those are not real children. They are technically either in shadow dom or emulated shadow dom. To gain access to those, you would need to use a ref.

@EisenbergEffect I agree on usage of child or children, but I need a definitive answer on whether or not it is possible to detect changes on custom element contents.
I developed several custom attributes, where I used MutationObservers to detect changes in html elements (and children) where I applied my attribute.
For my jstree wrapper I decided to go with a custom element, because I wanted to support several initialisation scenarios like dynamic html content (the above scenario), binding to a data attribute, binding to a callback function, etc.
When I tried a MutationObserver on src element (see my first post) to detect changes in the slot content, it didn't work. According to the official docs the ref attribute is not supported on slot element, so I tried wrapping it in an external div, and then I setup a MutationObserver on that wrapper div. But no events are raised on the observer when the ul content changes.
Now my first question is: is it expected that MutationObservers don't work in the above scenario? If it's so, my second and more important question is: How can I observe changes in custom element's content?
Thx

@nexbit Please post your code here or make a gist.

@levan-g here it is: https://gist.github.com/nexbit/eb207b12c16d74555076330278a4d409
pretty basic stuff, put that files under the skeleton-esnext directory of the aurelia skeleton, replacing existing welcome files.
If you try adding nodes the observer doesn't detect any change, but if you fiddle with the attributes of the root div (the src one), the observer will capture the attributes event type (you'll see logged in the console).
Obviously that is a stupid proof of concept, my real need of observing changes in a hidden div is to use the innerhtml to init/update a JQuery component that I host in the element div.
Now it's working very well by dirty checking the innerhtml of the src div, and updating the JQuery component when I detect any change. But you'll agree that a MutationObserver is way more efficient and less hacky..

authors: EisenbergEffect levan-g nexbit

repo: aurelia/templating


text:
Present version as 'UNKNOWN' if not found in database.

Older versions will not have the current_version field in the database so present it as unknown.

The latest changes are outside of the scope of the original issue that this is intended to solve. Is there a particular conversation or issue that you're looking to fix? or is this just preference?

@lszanto Let's try to just solve #194 by stating that the version is older than Canvas 2.1.7 if the value does not exist in the database. Could you refactor this PR?

@austintoddj I've changed it to the specific message however i'm not sure how to remove the user account commits. I forgot to switch to my testing branch.

Well, there's a few ways you could go about it, but you could just create a new PR with the specific changes, and then I could just close this one out. Your choice.

authors: austintoddj jlaswell lszanto

repo: austintoddj/canvas


text:
🛑 repl.co is down

In c2a6c74, repl.co (https://hello-repl.auteen.repl.co) was down:

HTTP code: 404
Response time: 18218 ms

Resolved: repl.co is back up in c0bd9d3.

repo: auteen/autoreplit


text:
Version 7.2.0 Will not minify properly.

My builds started failing today and I have traced it all the way back to node-jsonwebtoken: v7.2.0
Pinning my project to v7.1.10 minifies just fine.
Using:

gulp-uglify: 2.0.0

Have you tried v7.2.1? https://github.com/auth0/node-jsonwebtoken/blob/master/CHANGELOG.md#721---2016-12-07
There was a ES5 compatibility break in v7.2.0

@ziluvatar 7.2.1 seems to minify correctly. Thanks.

authors: failpunk ziluvatar


text:
Aut 157/feature/fe organiser home page

https://authenticket2023.atlassian.net/browse/AUT-157?atlOrigin=eyJpIjoiYzYyNDFhYTlkMGQ5NGM2OGE0ZjU0MDRhNDJjYzAzNjQiLCJwIjoiaiJ9

@nafutofu , please review and merge, thanks


text:
How to turn on the logs?

I saw some code in the provider file.
from opal_common.logger import logger
logger.debug('...')

I also tried to add some
print('...')

However, they never got printed after I docker compose up --build.
How can I turn on the logs? Thanks!

Hi @Hongbo-Miao ,
Check out the config flags  (starting with LOG_) at opal_common/config.py
(Also remember to prefix with OPAL_ when defining as env-vars)
You can see that the default log level is INFO
    #  - log level
    LOG_LEVEL = confi.str("LOG_LEVEL", "INFO")

and so only logs equal or higher are logged, debug is below and hence ignored.

Thanks @orweis , adding OPAL_LOG_LEVEL=DEBUG makes logger.debug('...') works!
I am surprised my print('...') not print.

Regarding the print - the server config is to propagate out stderr , and not stdout (you can change print of course see - https://www.systutorials.com/how-to-print-a-line-to-stderr-and-stdout-in-python/)

Oh I see, thanks for the info, @orweis !

Low key, check out loguru. It adds bloat to your project, but you have easy global controls in libraries using the configure, enable and disable commands.

authors: Hongbo-Miao kivo360 orweis

repo: authorizon/opal-fetcher-postgres


text:
segmentation fault when loading surrogate_model.model

I'm running a segmentation fault error when loading surrogate_model.model from nb311-v0.5/ensemble_member_0
Anyone has the same problem?

Hi, try these steps:
Check if you have nasbench installed. If not, install tensorflow==1.15.4 using pip, and then install nasbench following their installation instructions. Note: if you have tensorflow 2, or no tensorflow at all, then the nasbench installation will fail because it was created before tensorflow 2.
Once tensorflow 1.15.4 and nasbench are installed, try to load surrogate_model.model again. We will soon update the installation instructions.

It works now when I downgrade tf to the version you specified. Thanks!


text:
Update the Examples

This is related to #376. Example notebooks have to be updated to current versions of Talos.

I'm glad this issue is assigned. I found these problems so far :

In Comprehensive example, in order to import the datasets, we have to import from this path.
from talos.templates import datasets
In the talos.Scan parameters, the parameter dataset_name as in :
h = ta.Scan(x, y, params=p,          model=iris_model,          **dataset_name**='iris',          experiment_no='1',          grid_downsample=.01)
needs to be changed to experiment_name,
experiment_no  seems to have been removed
grid_downsample  is now fraction_limit
Thanks

@MorissetteThomasV do you want to make a PR for it? That would be great :)

authors: MorissetteThomasV mikkokotila

repo: autonomio/talos


text:
neus-acc with feature grids and  hash encoding

Hi,
I am trying to train dtu-scan65 dataset with neus-acc and hash encoding options.
the normal and the depth map are not properly estimated.
Is there any options that I have to set to the neus-acc?
In my opinion, pruning occupancy grid have a bad effect on the training based on hash grid encoding..

This is the command I have tried.
ns-train neus-acc --pipeline.model.sdf-field.inside-outside False --pipeline.model.sdf-field.use-grid-feature True --pipeline.model.sdf-field.encoding-type hash --pipeline.model.sdf-field.num-layers 2 --vis tensorboard  --data data/sdfstudio-demo-data/dtu-scan65

@slpmee The mode is sensitive to some hyper parameters beta-init and bias. You could try with
ns-train neus-acc --pipeline.model.sdf-field.inside-outside False --pipeline.model.sdf-field.use-grid-feature True --pipeline.model.sdf-field.encoding-type hash --pipeline.model.sdf-field.num-layers 2 --pipeline.model.sdf-field.beta-init 0.3 --pipeline.model.sdf-field.bias 0.5 --vis wandb --trainer.steps-per-eval-image 500 --data data/sdfstudio-demo-data/dtu-scan65

authors: niujinshuchong slpmee

repo: autonomousvision/sdfstudio


text:
feat(pose_initilizer): support gnss/imu pose estimator

Description
This is a modification of pose_initilizer to use GNSS/IMU pose estimtator.
When parameter ndt_enabled is false, LocalizationTriggerModule without ndt_trigger_ is activated.
Related discussions are here.
https://github.com/autowarefoundation/autoware.universe/pull/2848#discussion_r1104149473
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

This implementation was not adopted due to cyclomatic complexity
https://github.com/autowarefoundation/autoware.universe/pull/2904/commits/afdadb325b2ed66af12a4c788312162858c150f1

@YamatoAndo
Hello. I have extended pose_initilizer for gnss/imu pose estimator, please review.

@rsasaki0109
Is the copyright 2022 intentional? It is recommended to use the same year if it is almost the same as the original file, and 2023 if there is a lot of new code.

Just conforming to localization_trigger_module.cpp, no deeper meaning. I think your suggestion is good and I will modify it.

Is the copyright 2022 intentional? It is recommended to use the same year if it is almost the same as the original file, and 2023 if there is a lot of new code.

Sorry, this was a comment on the new file. Existing files will retain their original publication year. So please don't change ekf_localization_trigger_module.hpp.

@YamatoAndo @KeisukeShima
I would appreciate a review when you are free.

@YamatoAndo @Igata-ctrl
I don't have write permission, so could you please merge them?


text:
Move drivers into separate .repos file

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description
This issue is about moving the driver packages to a separate driver.repos file from autoware.repos file.
Purpose
Since not every vehicle will use all the drivers, it's better for them to have a separate driver.repos file.
This way when people fork autoware, it's easier for them to disable these repositories.
Possible approaches
We should simply separate the driver repositories.
But we will probably need to update the CI and the documentation for installation.
Current drivers:
  sensor_component/external/tamagawa_imu_driver:
    type: git
    url: https://github.com/tier4/tamagawa_imu_driver.git
    version: ros2
  sensor_component/external/velodyne_vls:
    type: git
    url: https://github.com/tier4/velodyne_vls.git
    version: tier4/universe
  vehicle/external/pacmod_interface:
    type: git
    url: https://github.com/tier4/pacmod_interface.git
    version: main

Definition of done

[ ] pacmod_interface is moved out
[ ] velodyne_vls is moved out
[ ] tamagawa_imu_driver is moved out
[ ] Documentations are updated
[ ] CI is updated

@xmfcx I'm sorry for joining late. Explaining the background, these are from TIER IV's reference vehicle/sensor_kit.
I think splitting repos or just removing some is okay, but there are several things to be discussed:

Which perspective to divide

Do you really want to split drivers? Or do you want to split the entire sensor components? I'm not sure which is better.

Since not every vehicle will use all the drivers, it's better for them to have a separate driver.repos file.

Seeing this purpose, I feel like you want to split the sensor components that aren't related to your vehicle.

This way when people fork autoware, it's easier for them to disable these repositories.

For this, I think users can do it with the current configuration because they are under the sensor_component section. :thinking:
Could you please show me some concrete examples?

Dependency

We might need to fix some dependencies before splitting them out.

Anyway, I'd like to clarify the background and issues more.
Honestly, I feel that the current proposal might be a bit short-sighted.
So could you share your troubles in more detail with me?

@xmfcx
Could you share more detail about the purpose of this issue?

Is it to create new repos file to list up Autoware-supported sensor drivers?, or
Do you just want to remove the need of installing unnecessary drivers?

If  latter is the case, I think we can just remove Tamagawa-imu and pacmod-interface drivers since they are not used in tutorials anyways.

So, I have thought about this more generally and here are the things I'd like to achieve:

Users shouldn't be forced to build drivers that they won't use.
The drivers should be tested in the repository through the CI to keep compatible.
Some drivers are essential, like the nebula driver.

For this, I will separate out the following:

pacmod_interface
tamagawa_imu_driver

into a different .repos file called extra-packages.repos.
Will modify the CI to pull them along with the others, build and test as usual.
The users will download the usual autoware.repos without these packages.
If they want to, they can download it with the extra-packages.repos.
In the future, we can think of similar non-essential external packages to this list.
Do you think this is reasonable? @esteve @mitsudome-r ?

@xmfcx I agree with you, the drivers are quite specific to the vehicle where Autoware is running, so it'd be better to move them to a separate .repos file and let users use whatever driver they need for their vehicle.

authors: esteve kenji-miyake mitsudome-r xmfcx

repo: autowarefoundation/autoware


text:
feat: sample mobile robot

Description
Create config files for sample mobile robot
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

Close due to incorrect merge destination.


text:
How can I keep the size of an image in the cache, when I updated an image?

Hello
how to keep the image size if it is lower compared to the filter the image of folder avalanche.

Dude, it's a deprecated Bundle. Sorry :/

authors: ClementRoupioz Jeff-GH


text:
[feature request] Keep full image/ aspect ratio on filters

Sorry if this isn't the right place for this.  But I haven't seen or figured out a way to do this.  I'd like to create a filter that sets the max width, and resizes solely on that value.  The goal would be to keep the full image and aspect ratio, and resize it down based on the width or height specified.

LiipImagineBundle has support for this:
https://github.com/liip/LiipImagineBundle/blob/master/Imagine/Filter/Loader/ThumbnailFilterLoader.php
In your filter config you can optionally specific null for width or height

Not to steal @lsmith77's thunder, but I ended up implementing the same thing in JmikolaImagineBundle after a discussion with Bulat about adding to this bundle. In fact, the methods for these operations already exist in Imagine - they just weren't available as configurable filters.

best would be to combine it all #liip/LiipImagineBundle/8

authors: JasonHears jmikola lsmith77


text:
Fix checkbox display

Using form-control on "input type=checkbox" breaks normal rendering and behavior in Chrome (didn't test FF).
The class form-control applies the CSS rule appearance:none in this line:
https://github.com/almasaeed2010/AdminLTE/blob/master/dist/css/AdminLTE.css#L1272
If you check the sources at:
https://almsaeedstudio.com/themes/AdminLTE/pages/forms/general.html
you will see, that checkboxes are not meant to use the form-control class.

Thanks!

repo: avanzu/AdminThemeBundle


text:
Create Median of Two Sorted Arrays

Median of 2 sorted array.

@mrgokuji this program is already added in c++ , choose another language.
create an issue again , closing the pr :)


text:
[DOCS] Convert markdown issue templates to issue forms

Description
Completing an issue is very hard for new contributors in markdown format, since many of them are unaware, and it gives lot of flexibility.

Making issue forms is better for good managing of issues, as it is of the same structure.

I will also add a proper checklist with standard conventions
Screenshots
No response
Additional information

GSSoC'23.

Code of Conduct

[X] I agree to follow this project's Code of Conduct

Go Ahead @Anmol-Baranwal

authors: Anmol-Baranwal avinashkranjan


text:
[ADDED]: PhoneSploit Script

Description
The PhoneSploit Script has been added in the repo.
Fixes #303
Have you read the Contributing Guidelines on Pull Requests?

[X] Yes
[ ] No

Type of change

[X] New feature (non-breaking change which adds functionality)

Checklist:

[X] My code follows the style guidelines(Clean Code) of this project
[X] I have performed a self-review of my own code
[X] I have commented my code, particularly in hard-to-understand areas
[X] I've edited the SCRIPTS.md and link to my code.
[X] I have created a helpful and easy to understand README.md
[X] My documentation follows Template for README.md
[X] My changes generate no new warnings

@avinashkranjan Can you please review my PR and kindly merge it :)


text:
Initial sample app

Let me know if you think I should add a more complex example that uses a custom implementation of AdapterPresenter or some other features

@mcasper3 by the way, the images. Are they free to use?

@Kistamushken yes, the photos are from https://unsplash.com/

repo: avito-tech/Konveyor


text:
Links broken in "Usage" wiki

In the page "https://github.com/awaitility/awaitility/wiki/Usage", under the section Links and code examples, the links mentioned are broken.
Eg.
The link mentioned is:
http://github.org/awaitility/blob/master/awaitility/src/test/java/org/awaitility/UsingAtomicTest.java
but it should be:
https://github.com/awaitility/awaitility/blob/master/awaitility/src/test/java/org/awaitility/UsingAtomicTest.java
Kindly update. Unfortunately we can't create Pull request for Github Wiki, or else I would have fixed it myself.
Thanks for a wonderful library.

Thanks for point this out and for your kind words, I've fixed the links now.

authors: johanhaleby thsaravana

repo: awaitility/awaitility


text:
System Screen Capture glitches

Bug report, debug log and your config file (FULL LOGS ARE MANDATORY)
Debug log
Config file

Steps to reproduce

Go to remote control
Enable System Screen Capture
Check Live Video in the LED Visualization window

What is expected?
It should show the current screen image without gif
What is actually happening?
The image looks distorted or glitchy like in the attached picture. This also affects to the LEDs ilumination.

System
HyperHDR Server:
- Build:           (HEAD detached at d7e3117) (Awawa-bba94d0/d7e3117-1651692977)
- Build time:      May  4 2022 19:47:22
- Git Remote:      https://github.com/awawa-dev/HyperHDR
- Version:         18.0.0.0beta2
- UI Lang:         auto (BrowserLang: es-419)
- UI Access:       expert
- Avail Capt:      Windows (Microsoft Media Foundation)
- Database:        read/write

HyperHDR Server OS:
- Distribution:   Windows 11 Version 2009
- Architecture:   x86_64
- Kernel:         winnt (10.0.22000 (WS: 64))
- Qt Version:     6.2.4
- Browser:        Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/101.0.4951.67 Safari/537.36

I restored you configuration and test it: works fine.
Since your path is clear but since I can't reproduce the problem I can' fix it. Maybe it's a matter of your graphic card drivers (tested on GTX1060) or Windows 11 (the screen grabber was designed for DX11, the compatibility should be sustained, I can guarantee it for MS).

I restored you configuration and test it: works fine. Your path is clear but since I can't reproduce the problem I can' fix it. Maybe it's a matter of your graphic card drivers (tested on GTX1060) or Windows 11 (the screen grabber was designed for DX11, the compatibility should be sustained, I can't guarantee it for MS).

In my case I'm using a RTX2060 connected to an Alienware 17 R3 laptop using an Alienware Graphics Amplifier, so maybe the problem is with my particular setup. What's strange to me is that using the Hyperion Screen Capture software and sending the image to the HyperHDR server with Flatbuffers works, but I think this is more CPU consuming, that's why I wanted to make the native screen capture works. By the way, I have the same problem using v17 and Hyperion.

The problem should be fixed now.
You can find the installer in the latest github action after the build is finished (currently: https://github.com/awawa-dev/HyperHDR/actions/runs/2393373150 )
Let me know if it solved your problem.
You need to check that option:

It worked! Thank you so much!

authors: awawa-dev grigosback

repo: awawa-dev/HyperHDR


text:
Can it convert this format?

I'm using https://github.com/dtinth/promptpay-qr to generate the qr code value that ends up as shown below, it doesn't seem to display properly using this library, just shows as random dotted lines, is this kind of data supported please?
Value: data:image/svg+xml,%3Csvg%20xmlns%3D%22http%3A%2F%2Fwww.w3.org%2F2000%2Fsvg%22%20viewBox%3D%220%200%2037%2037%22%3E%3Cpath%20fill%3D%22%23ffffff%22%20d%3D%22M0%200h37v37H0z%22%2F%3E%3Cpath%20stroke%3D%22%23000000%22%20d%3D%22M4%204.5h7m2%200h3m1%200h3m2%200h1m3%200h7M4%205.5h1m5%200h1m4%200h3m1%200h5m2%200h1m5%200h1M4%206.5h1m1%200h3m1%200h1m2%200h3m1%200h1m8%200h1m1%200h3m1%200h1M4%207.5h1m1%200h3m1%200h1m1%200h1m1%200h2m1%200h2m2%200h1m2%200h1m1%200h1m1%200h3m1%200h1M4%208.5h1m1%200h3m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h3m1%200h1M4%209.5h1m5%200h1m2%200h2m2%200h1m5%200h1m2%200h1m5%200h1M4%2010.5h7m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h1m1%200h7M13%2011.5h1m2%200h1m1%200h1m2%200h3M4%2012.5h2m3%200h3m1%200h2m1%200h2m2%200h1m1%200h1m1%200h1m3%200h2M4%2013.5h1m6%200h2m1%200h1m3%200h2m1%200h2m2%200h2m5%200h1M5%2014.5h4m1%200h2m1%200h1m1%200h2m2%200h3m1%200h1m1%200h1m2%200h1m1%200h2M5%2015.5h2m1%200h1m2%200h1m2%200h2m6%200h2m4%200h1M4%2016.5h1m1%200h2m2%200h5m2%200h2m1%200h1m1%200h5m1%200h1m3%200h1M4%2017.5h4m1%200h1m1%200h1m1%200h2m1%200h1m1%200h1m1%200h1m7%200h2m2%200h1M7%2018.5h2m1%200h5m2%200h1m1%200h1m1%200h1m2%200h2m5%200h1M4%2019.5h4m1%200h1m1%200h1m1%200h1m2%200h3m1%200h1m2%200h1m2%200h1m2%200h1m1%200h1M5%2020.5h1m3%200h5m2%200h2m2%200h1m2%200h1m1%200h1m2%200h2m1%200h1M4%2021.5h1m1%200h3m2%200h4m3%200h1m1%200h2m1%200h2m2%200h2M4%2022.5h2m2%200h4m1%200h2m1%200h3m1%200h2m1%200h7m1%200h2M4%2023.5h1m1%200h1m4%200h2m2%200h1m5%200h2m5%200h2M4%2024.5h1m2%200h1m1%200h5m1%200h1m3%200h1m2%200h8m1%200h1M12%2025.5h3m1%200h3m2%200h2m1%200h1m3%200h1m3%200h1M4%2026.5h7m1%200h1m4%200h3m2%200h3m1%200h1m1%200h1m1%200h2M4%2027.5h1m5%200h1m1%200h2m2%200h2m3%200h2m1%200h1m3%200h4M4%2028.5h1m1%200h3m1%200h1m3%200h4m2%200h9m3%200h1M4%2029.5h1m1%200h3m1%200h1m2%200h2m4%200h1m2%200h1m5%200h1M4%2030.5h1m1%200h3m1%200h1m2%200h1m1%200h2m5%200h1m2%200h2m1%200h1m2%200h1M4%2031.5h1m5%200h1m1%200h1m2%200h1m6%200h3m2%200h2m2%200h2M4%2032.5h7m1%200h2m1%200h1m3%200h2m2%200h1m1%200h2m1%200h1m2%200h1%22%2F%3E%3C%2Fsvg%3E

Do you want to present a QR Code of your base64 string or the value from the QR Code generated by the base64 string? Either way, I think it's all working fine. Please let me know more detail about your issue. :)

QR code please, mine is looking like the first image, how did you get it to look like the second?

It seems that you have a svg file format string. Maybe you can use this https://github.com/target/react-native-svg-parser to convert it into react-native-svg compatible components.
FYI, the second one is derived from the value of the SVG image directly.

authors: MrShakes awesomejerry

repo: awesomejerry/react-native-qrcode-svg


text:
Opening .ssa file

Hi there,
I am trying to open an exported extraction from a sono-trial, which is saved as an .ssa file. I checked the version of it in the textfile, 3.10, and the sampling frequency is at around 520 Hz. The .ssa-file also does not include more than 50'000 data points so i dont know why i keep getting the error message that it could not parse the .ssa file. Thank you already in advance for your h

elp.

Asarah94,
This an error from sonometrics software, nothing is to do with sonocleaner.  There problem is total rows are not match between the ssa header file and the total actual row in the ssa file.  Please edit the row number reports in ssa header file which will solve your problem.
Hoa

repo: awjchen/SonoCleaner


text:
This bot triggers a warning when using the "comment and close" feature of github

See https://github.com/aws/aws-sdk-php/issues/2310#issuecomment-915164704 for such a case

OK, this is actually intended to be commented on all closed issues at the time when they are closed, not in reaction to someone commenting on the closed issue. So maybe just a question of wording (but I have no idea of a better wording)

repo: aws-actions/closed-issue-message


text:
Dynamic Group Auth Not Being Respected by Subscription

How did you install the Amplify CLI?
pnpm
If applicable, what version of Node.js are you using?
No response
Amplify CLI Version
11.0.0
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made
Describe the bug
This blog post (https://aws.amazon.com/blogs/mobile/build-real-time-multi-user-experiences-using-graphql-on-aws-amplify/) made me think that Subscriptions might respect dynamic group authorization, but based on my testing my onCreate subscription does receive every notification no matter which group it belongs to (my user only belongs to one group).
If this is the case, doesn't this allow an user to modify the frontend code, create subscriptions and receive updates of anything, no matter if the user is in the same group or not? Unless Subscriptions are explicitly disabled of course, which is not by default.
Expected behavior
Subscription respects group auth rule
Reproduction steps

Create and deploy schema

type Notification
  @model
  @auth(
    rules: [
      { allow: private, provider: iam }
      { allow: groups, groupsField: "group" }
      { allow: public, operations: [] }
    ]
  )
  @key(
    name: "byGroupByStatus"
    fields: ["group", "status"]
    queryField: "listNotificationsByGroupByStatus"
  )
  @key(
    name: "byGroupByCreatedAt"
    fields: ["group", "createdAt"]
    queryField: "listNotificationsByGroupByCreatedAt"
  ) {
  id: ID!
  type: NotificationType!
  status: NotificationStatus
  group: String!
  reservationId: ID!
  createdAt: AWSDateTime! # Put here to use as a index (Amplify auto-generates this field too)
}

enum NotificationStatus {
  UNREAD
  READ
}
enum NotificationType {
  EXTERNAL_NOTE
}

subscribe to data as user that belongs to group
create record from another user
observe update

Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

Hi @chrisbonifacio .
Here are the headers for the mentioned request:

And here is the initial onUpdateNotification message on the WS, which appears to have some access token.  Is this enough to confirm Cognito was used?

Also when I'm logging the subscriptions, provider here has Cognito in the _config listed as authentication type.
    const subscription = API.graphql(
      graphqlOperation(subscriptions.onUpdateNotification)
    ).subscribe({
      next: ({ provider, value }) => console.log({ provider, value }),
      error: (error) => console.warn(error),
    });

// provider._config: aws_appsync_authenticationType: "AMAZON_COGNITO_USER_POOLS"

@parvusville still in the process of reproducing this issue. From the schema it looks like you're using the V1 GraphQL transformer. Is there a reason why you haven't upgraded to V2?

Hello @chrisbonifacio . In short I did try to upgrade, but there still seem to be some problems with it. Currently I am working on implementing a true multi-tenancy, and I couldn't use more time on figuring out the problems with the new version.
Some examples of why I have not upgraded:
My initial issues:  https://github.com/aws-amplify/amplify-category-api/issues/789
Later I have also seen that quite a few other problems exist, that seem to be at least somewhat related to dynamic group authentication.
This one in particular worries me: https://github.com/aws-amplify/amplify-category-api/issues/110
Probably related: https://github.com/aws-amplify/amplify-category-api/issues/82
Another one with a workaround causing more technical debt: https://github.com/aws-amplify/amplify-category-api/issues/61

HI @parvusville, as mentioned by @chrisbonifacio, the functionality for dynamic subscription filtering (as documented in that blog post) is only available for GraphQLv2 Backends, and there are no plans to back-port functionality to V1. I'd suggest migrating to V2 to continue receiving new functionality as it is released, as the only changes we're making to V1 moving forward will be security fixes. Resolving this ticket as question answered.

authors: alharris-at chrisbonifacio parvusville

repo: aws-amplify/amplify-category-api


text:
AmplifyGraphqlApi: latest release for creating a resolver not working (detailed below)

How did you install the Amplify CLI?
No response
If applicable, what version of Node.js are you using?
No response
Amplify CLI Version
2.100.0
What operating system are you using?
Ubuntu 23.04
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
N/A
Describe the bug
Example found here missing something--https://aws.amazon.com/blogs/mobile/announcing-aws-amplifys-graphql-api-cdk-construct-deploy-real-time-graphql-api-and-data-stack-on-aws/
version: 2.100.0
In the backend stack, amplifyApi.addResolver is not identifying the following:
"Code.fromAsset"
and
"FunctionRuntime"
Reviewing the docs, this seems to be a new feature.
Expected behavior
Resolver is deployed correctly.
Reproduction steps
Work thru the example given on the blog.
Project Identifier
No response
Log output

# Put your logs below this line

Additional information
No response
Before submitting, please confirm:

[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

This issue has been resolved and updated in the CDK repository's issue.


text:
Exception while upgrading the amplify cli.

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.
[X] I have removed any sensitive information from my code snippets and submission.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
v16.14.0
Amplify CLI Version
8.5.0, 8.4.0, 8.3.0
What operating system are you using?
Amazon Linux 2 on Cloud9 instance.
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made.
Amplify Categories
Not applicable
Amplify Commands
Not applicable
Describe the bug
I'm getting lately this error when I upgrade the amplify cli:
`Developer2:~/environment/xxxxx (xxxxx) $ amplify upgrade
Downloading latest Amplify CLI
100% [========================================] 0.0 seconds left
⠸ Validating downloaded Amplify CLI....internal/modules/cjs/loader.js:905
throw err;
^
Error: Cannot find module '/home/ec2-user/environment/xxxx/--version'
at Function.Module._resolveFilename (internal/modules/cjs/loader.js:902:15)
at Function._resolveFilename (pkg/prelude/bootstrap.js:1819:46)
at Function.Module._load (internal/modules/cjs/loader.js:746:27)
at Function.runMain (pkg/prelude/bootstrap.js:1847:12)
at internal/main/run_main_module.js:17:47 {
code: 'MODULE_NOT_FOUND',
requireStack: []
}
✖ An unexpected error has occurred, opt in to send an error report to AWS Amplify with non-sensitive project configuration files. Confirm  (y/N)`
The last line is new, it was first stuck showing waiting for validation, after a control-c, the "An unexpected..." message appeared, but I could not select the confirm, the script was not running anymore.
The version is now 8.5.0, prior was 8.4.0, which had the same error, but the version seems to work regardless of the exception.
It's kind of annoying though, I always need to cross fingers when I upgrade the cli.
Expected behavior
Clean upgrade with no issues.
Reproduction steps
I can't provide steps to reproduce, no time currently to do so.
GraphQL schema(s)

# Put schemas below this line

Log output

2022-06-14T15:01:53.686Z|info : amplify upgrade core
2022-06-14T15:01:53.736Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2022-06-14T15:06:52.247Z|info : amplify version core  {"version":true,"yes":false}
2022-06-14T15:07:02.753Z|info : amplify status core  {"yes":false}
2022-06-14T15:07:02.806Z|info : amplify-cli-core.banner-message/index.ts.fetch banner messages from https://aws-amplify.github.io/amplify-cli/banner-message.json({}
2022-06-14T15:07:04.511Z|info : amplify-provider-awscloudformation.display-helpful-urls.showCognitoSandBoxMessage(["userPool"])

Additional information
The log does not show any errors, see above.

I noticed, that I'm getting the following weird messages on push with the new version 8.5.0, there where apparently fixed in 8.4.0, but appeared also on 8.3.0 as I remember.
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
Overrides functionality is not implemented for this category
✔ Successfully pulled backend environment staging from the cloud.
Overrides functionality is not implemented for this categor```

Hey @ecc7220 :wave: thanks for raising this! This is a duplicate bug of https://github.com/aws-amplify/amplify-cli/issues/10583#issuecomment-1154508064

There was a bug in amplify upgrade in amplify-cli 8.4.0 which has been fixed in amplify-cli 8.5.0 . Please install the CLI using npm install -g @aws-amplify/cli@8.5.0 . The problem in amplify upgrade will not be seen in future versions of the CLI. Closing this, since resolved in 8.5.0.

And for your second note this is documented as an active bug here https://github.com/aws-amplify/amplify-cli/issues/10546
Closing in favor of the explanation on #10583 and duplicate bug of #10546

repo: aws-amplify/amplify-cli


text:
Error when trying to install v1.11.0 in Ubuntu

Note: If your issue/bug is regarding the AWS Amplify Console service, please log it in the
Amplify Console GitHub Issue Tracker

Describe the bug
When I do npm i -g @aws-amplify/cli it ends with these errors:
gyp WARN EACCES user "nobody" does not have permission to access the dev dir "/home/******/.cache/node-gyp/11.14.0"
gyp WARN EACCES attempting to reinstall using temporary dev dir "/usr/lib/node_modules/@aws-amplify/cli/node_modules/kerberos/.node-gyp"
gyp WARN install got an error, rolling back install
gyp WARN install got an error, rolling back install
gyp ERR! configure error
gyp ERR! stack Error: EACCES: permission denied, mkdir '/usr/lib/node_modules/@aws-amplify/cli/node_modules/kerberos/.node-gyp'
gyp ERR! System Linux 5.0.0-23-generic
gyp ERR! command "/usr/bin/node" "/usr/lib/node_modules/npm/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /usr/lib/node_modules/@aws-amplify/cli/node_modules/kerberos
gyp ERR! node -v v11.14.0
gyp ERR! node-gyp -v v5.0.3
gyp ERR! not ok

Even when trying the command with sudo it doesn't work.
To Reproduce
Steps to reproduce the behavior:

Install @aws-amplify/cli@1.11.0 on Ubuntu
See error

Expected behavior
A clear and concise description of what you expected to happen.
Screenshots
If applicable, add screenshots to help explain your problem.
Desktop (please complete the following information):

OS: Ubuntu 19.04

Additional context
Installing 1.10.0  works fine

What’s the error that you get with sudo?

Same error

Did a bit more searching and apparently as per #943 if I add in the --unsafe-perm flag after sudo, it installs fine.
Not sure if I should keep this open @kaustavghosh06

@nino-moreton Yes, I've seen using npm install -g @aws-amplify/cli@beta --unsafe-perm=true make the installation work (and resolve the gyp issues) for certain npm versons  installed.


text:
Error in Amplify CLI's latest(4.24.0) version in Build Image: auth headless is missing the following inputParams facebookAppIdUserPool, facebookAppSecretUserPool, googleAppIdUserPool, googleAppSecretUserPool

Is your feature request related to a problem? Please describe.
I'm configuring Continuous-Deployment with this repository's temporary feature branch and I was working react frontend UI.
I pushed some commits that are not related to amplify's backend environment.
after checking in Amplify console, it fails in the build step!
It just passed several days ago.
The detailed error message was...
[INFO]: auth headless is missing the following inputParams facebookAppIdUserPool, facebookAppSecretUserPool, googleAppIdUserPool, googleAppSecretUserPool
[INFO]: Error: auth headless is missing the following inputParams facebookAppIdUserPool, facebookAppSecretUserPool, googleAppIdUserPool, googleAppSecretUserPool
                                 at updateConfigOnEnvInit (/root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-category-auth/provider-utils/awscloudformation/index.js:370:15)
                                 at /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/amplify-category-auth/index.js:240:28
                                 at /root/.nvm/versions/node/v10.16.0/lib/node_modules/@aws-amplify/cli/node_modules/promise-sequential/index.js:16:18
                                 at process._tickCallback (internal/process/next_tick.js:68:7)
[ERROR]: !!! Build failed

I tried to change the shell script to execute headless cli refer to #2877.

but no luck! and How can I find the documentation of the headless cli's categories parameter? I can't find in any documentation

Describe the solution you'd like
I was set up the amplify cli version in build image to latest.
The version I failed is 4.24.0.
so changing cli version from latest to 4.20, error disappeared and passed the all steps!
(version 4.20 is the version in my local environment)
Additional context
so Here are the questions:

How can I use the headless cli with handling the following inputParams to pull backend environment in latest cli version(4.24.0)?

inputParam like facebookAppIdUserPool, facebookAppSecretUserPool, googleAppIdUserPool, googleAppSecretUserPool
is there any documentation or change log?

please let me know. that kind of undocumented issues make me exhausted...

Hello @geoseong
How did you change the shell script? If you are using Amplify Console for your builds you would need to include those input params as environment variables.  An example of this is noted in this comment: https://github.com/aws-amplify/amplify-cli/issues/4790#issuecomment-656422252
Reference from Amplify Console Repo: https://github.com/aws-amplify/amplify-console/issues/206#issuecomment-657013051
Closing this as a duplicate of #4790.

authors: SwaySway geoseong


text:
Adding a layer resets function timeout

Hi all - I'm noticing that my timeouts are all getting reset back to 25 seconds on my Lambdas after adding a layer. Learned the hard way today. :( Hope others can learn from this.

It should be possible to configure timeout per function when creating/updating the function
It should be possible to select the wanted Node version. It's currently defaulting to 10.x when 12.x is already available in Lambda
It should be possible to configure RAM available to the function

A fix was merged and deployed in Amplify CLI version 4.26.0.

authors: houmark jhockett kennandavison


text:
AWS Amplify Using Subscription Method with DynamoDB (An error occurred when pushing the resources to the cloud)

Hi there!
I am working on project with react native ans AWS Amplify, I am using graphQL API with DynamoDB
I am getting a subscribe return from any changes happen in Table Post, I need to subscribe only when the post identify by user is change, so I am trying to create a filter to Subscription method using that code below and got an error:
the code I use
type Subscription {
  onUpdatePost(userId: ID!,  deleted: Boolean!): Post
    @aws_subscribe(mutations: ["updatePost"])
}

the error I get
✖ An error occurred when pushing the resources to the cloud

Object type extension 'Subscription' cannot redeclare field onUpdatePost
An error occurred during the push operation: Object type extension 'Subscription' cannot redeclare field onUpdatePost

@MedNabilEssefaihi this is probably because the @model directive is generating a conflicting onUpdatePost operation. You can turn this off by adding @model(subscriptions: null) to the directive. Note that this will also remove automatic generation of update and delete subscriptions as well.

authors: MedNabilEssefaihi edwardfoyle