text:
Detect test dependencies that are mistakenly added to non-test configurations

Is your feature request related to a problem? Please describe.
Sometimes a developer accidentally adds a test dependency to release or production code.
e.g.
   // whoops, should have been  testImplementation(project(":test-support:my-test-lib"))
   implementation(project(":test-support:my-test-lib"))

   // whoops, should have been  testImplementation("org.junit:junit:4.13.2")
   implementation("org.junit:junit:4.13.2")

This bloats release code and can cause various mysterious issues can be hard to spot...
Describe the solution you'd like
Detect when people write implementation when they should have used testImplementation instead.
This can be hard to detect automatically...
because if you are writing a testing lib, then implementation("org.junit:junit:4.13.2") is correct!
But you could look for deps that are included directly in both test and non-test contexts, which might signal there is a problem.
Describe alternatives you've considered
Might need an API to signal if a project is intended to be used as a test dep.
my-test-lib/build.gradle.kts ->
dependencyAnalysis {
     intendedToBeATestDep()
}

I think the plugin already handles the following case (pseudo-code):
// build.gradle
implementation junit4

// src/test/SomeTest.java

public class SomeTest {
  @Test void test() ...
}

And because JUnit's @Test annotation is only used in a test source set, the plugin will say you can change junit from implementation to testImplementation.
Are you saying it should go further and flag dependencies that are only meant to be used by tests but are nevertheless on the main compile classpath? I think that would indeed require some kind of new DSL option(s).

Same logic works for project(...) deps? If so, that probably covers mostly what I was after here.
The other case is when something should ONLY be used in tests, possible because of license issues, or other reasons.
And a dev might add it as implementation in production code to use one method in there, and it wouldn't get flagged as it is actually being used. But we want to flag it because we intend for that project or dep to only ever be used in test configurations.

Can you tell me more about this license-based use-case? I want to
understand it more.
On June 29, 2021, GitHub @.***> wrote:

Same logic works for project(...) deps? If so, that probably covers
mostly what I was after here.
The other case is when something should ONLY be used in tests,
possible because of license issues, or other reasons.
 And a dev might add it as implementation in production code to use
one method in there, and it wouldn't get flagged as it is actually
being used. But we want to flag it because we intend for that project
or dep to only ever be used in test configurations.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub
<https://github.com/autonomousapps/dependency-analysis-android-gradle-
plugin/issues/434#issuecomment-871072785>, or unsubscribe
<https://github.com/notifications/unsubscribe-
auth/ABJG5PPEWNMU6ZXQVRGC7L3TVKHGJANCNFSM47RGMHEQ>.

There may be libraries with licenses that have different restrictions or obligations based how the library is used.
E.g. distribution of the library versus just using at build time.
So usage in tests may be compliant with the license, but accidentally distributing as part of a production release of software may have different implications under the terms of the license.
(I am not a lawyer or license expert, so take this worth a grain of salt)

Thanks, this is an interesting use-case. I don't think it's out of bounds, but I also can't claim that it's something I'll work on in the near future. If it's something you'd like to work on, we could have a conversation about how it might be implemented.
It might also be possible to bolt on this behavior via your own plugin or build script that parses the output of this plugin's tasks.

I think you're better off using something like https://github.com/dropbox/dependency-guard for your use-cases.

authors: autonomousapps yogurtearl


text:
Baseline support

Is your feature request related to a problem? Please describe.
In a big project it would be helpful to be able to generate a baseline to prevent future problems, and temporarily accept the current ones.
Describe the solution you'd like
gradlew buildHealthBaseline with a configurable location:
ideally there would be individual baseline files like in Android Lint:
dependencyAnalysis {
  baseline {
    allprojects.forEach {
      project(it.path) {
        baseline.set(it.rootDir.file("config/deps/baseline-${project.name}.xxx"))
      }
    }
  }
}

but a single file is also a win:
dependencyAnalysis {
  baseline.set(layout.projectDirectory.file("foo/bar"))
}

Describe alternatives you've considered
This is currently possible by listing them one by one, like:
project(":foo:bar:baz") {
  onUnusedDependencies {
    exclude(libs.android.guava.get().toString())
  }
}

sadly, this is very tedious, and also leaves more tech debt than before because of potentially unused configuration (#271), and it is sometimes hard to match the warning text to the DSL.
Additional context
See also #271 and mind version control (see https://github.com/autonomousapps/gradle-best-practices-plugin/issues/16)

@autonomousapps
The requested feature is useful for long living projects.
Just mentioning this in hope the priority of the feature would be increased

Unfortunately, I don't have time for things that aren't bug fixes or non-critical enhancements right now. I am open to sponsored work, however.

source: gharchive/issue

license_type: permissive

license_source: github-api


text:
Validation Loss Increasing

Hi authors, thank you for your great work.
I tried to train your network from scratch with ./configs/img/onet.yaml. I find something weird that the validation loss is increasing during training though the validation IOU is getting better. I also evaluated the model and the result looks fine(0.584 mean IOU). I am wondering if the validation loss would be normal.
Here is the screen shot for the loss curves.

@JiangWenPL Hi, have you solved the problem? I found same problems when i finetune the pretrained model on iGibson dataset, the training loss is declining while the validation loss is rising, though the validation IOU seems fine. At begining, I thought the network was overfitting, then I stop the iteration as early as possible, but it did little a help the evaluation result in test dataset.

authors: JiangWenPL fangchuan

license: MIT


text:
Can't get reasonable results from data that nerfstudio aces?

I've been struggling to attain reasonable results for my custom nerfstudio-like data, similar to https://github.com/autonomousvision/sdfstudio/issues/49, https://github.com/autonomousvision/sdfstudio/issues/85, etc., so I took a step back:  What if I just take nerfstudio's example "poster" dataset (which nerfstudio aces) and directly run sdfstudio on it? It turned out I was unable to get reasonable results for that dataset either.
Steps to reproduce the behavior:

Download nerfstudio's poster dataset.
Run, say, neus-facto with the nerfstudio-data data parser.
Look at the tensorboard results.

One example full command:
method=neus-facto
ns-train \
    $method \
    --pipeline.model.sdf-field.inside-outside True \
    --pipeline.model.background-model mlp \
    --output-dir sdfstudio_train/nerfstudio_data/poster/ \
    --method-name $method \
    --experiment-name sanity-check-mlp-bg-focus-vertical \
    --vis tensorboard \
    nerfstudio-data \
    --data nerfstudio/data/nerfstudio/poster \
    --center-method focus \
    --orientation-method vertical

I tried numerous combinations of:

--pipeline.model.background-model mlp
--pipeline.model.sdf-field.inside-outside True
Patching this PR: https://github.com/autonomousvision/sdfstudio/pull/90
--pipeline.model.sdf-field.use-grid-feature False
--pipeline.model.far-plane 2.0
--pipeline.model.sdf-field.num-layers 8
--pipeline.model.enable-collider False

Example failures:

What am I missing? Thanks in advance for thoughts/suggestions.

Thanks for the prompt help. I got similar results with your command on this poster scene. One quick follow-up question: Why --pipeline.model.sdf-field.inside-outside False? The CLI help said one should set it to True for indoor scenes.

I thought this is a 360 inward scene so I use --pipeline.model.sdf-field.inside-outside False where the wall of the room is modelled as background. Using --pipeline.model.sdf-field.inside-outside True might also works.

@xiumingzhang  I got simlilar error.  under same data, nerfacto can export correct pointcloud. however nues facto can't.
the pointcloud seems constitute an circle around center.
have you figure out the solution? many thx.

authors: fafancier niujinshuchong xiumingzhang

repo: autonomousvision/sdfstudio


text:
Help needed in matching a PCD map created within a simulator with a Lanelet2 data created in real-world

Checklist

[X] I've read the contribution guidelines.
[X] #1498
[X] I've agreed with the maintainers that I can plan this task.

Description
Basically, help from any member who is familiar with matching PCD data to lanelet2 data is needed to solve the following problem
A PCD map for ITRI Campus is created using MORAI SIM: Drive (Actually using AWSIM from TierIV as well).
I put the PCD map along with the original lanelet2 data in the same path.
When launching the Autoware using the command below, it is shown in the rviz that PCD data doesn't match the lanelet2 data. (It only shows PCD data. Lanelet2 data cannot be found)
ros2 launch autoware_launch autoware.launch.xml map_path:={the_path_including_pcd_map_and_lanelet2_data} vehicle_model:=sample_vehicle sensor_model:=sample_sensor_kit
The picture below shows that the PCD data is not displayed on top of the lanelet2 data. (Note the PCD map here is only a partial one by the way)

When using PCD data created in the real-world instead, the PCD map and lanelet2 data match altogether correctly.

Purpose
This task is required to have an E2E simulator (either MORAI SIM: Drive or AWSIM) prepared for bus ODD development & testing.
Related Issues in the Simulation WG

Integration of ITRI campus with ITRI bus and Autoware - MORAI SIM
Integration of ITRI campus with ITRI bus and Autoware - AWSIM

Possible approaches
I'm guessing there must be a way to specify how to convert the PCD data onto the global coordinate system so that it can be matched onto the lanelet2 map.
I'm currently looking into map_loader package, but I cannot find one yet.
I know how to match this PCD data with the lanelet2 data theoretically, but I don't know how to let Autoware do that.
Here, the PCD data is described w.r.t. simulation map coordinate system whereas lanelet2 data is done so w.r.t. the global coordinate system. Therefore I believe the problem is simply specifying the projection for the map coordinate system.
FYI: Matching the ITRI map coordinate system (simulation) with the global coordinate system
Definition of done
When launching the Autoware using the command below, rviz should show PCD data on top of lanele2 data.
ros2 launch autoware_launch autoware.launch.xml map_path:={the_path_including_pcd_map_and_lanelet2_data} vehicle_model:=sample_vehicle sensor_model:=sample_sensor_kit

@sglee-morai If you would like to move pointcloud you can either use pcl_tools or CloudCompare

pcl_tools: install sudo apt install pcl-tools and run  pcl_transform_point_cloud
CloudCompare: https://www.danielgm.net/cc/

@sglee-morai If you would like to move pointcloud you can either use pcl_tools or CloudCompare

pcl_tools: install sudo apt install pcl-tools and run  pcl_transform_point_cloud
CloudCompare: https://www.danielgm.net/cc/

I second pcl-tools. In the past I had a similar issue: I created a lanelet2 map from JOSM (georeferenced) and had to match it on the PCD (which was not). You can use the following principle:

Identify 2 points (or more) that are both on your lanelet and pcd. For each, find its global (latlon/mgrs/utm) and local (pcd) coordinates
compute global and local coordinate of the center point
compute average required rotation angle around the center point (stddev should be low)
then run something like:

# change pcd origin to center point
pcl_transform_point_cloud src dst -trans center_p["local"].x,center_p["local"].y,0
# rotate pcd around the center point
pcl_transform_point_cloud dst dst -axisangle 0,0,1,avg_rot
# translate pcd back to match global coordinates
pcl_transform_point_cloud dst dst -trans -center_p["global"].x,-center_p["global].y,0
# eventually you may have to change the format to binary. You can do this with some python:
# pcl.save(pcl.load(input), output, binary=True)

Eventually double check that each point you used as referenced have been moved at the expected coordinate.

@mitsudome-r @VRichardJP Thanks a lot for providing highly detailed guidelines!
PCD map sample from the simulator & ITRI Campus Lanelet2 Map captured in the real-world campus
Here is a download link for the initial sample PCD map, created using MORAI SIM: Drive, that you might take a look at if you'd like.
Btw, the real-world lanelet2 data that I'm trying to match the PCD map to is the ITRI campus lanelet2 map uploaded in the CI/CD pipeline. (I believe I'm not able to share it publicly.)
Matching
I'm currently trying it out and it seems like I'm almost there. (Blue one is from the real-world one, whereas the red one is from MORAI SIM: Drive)
Due to other issues, I wasn't able to follow the way that  @VRichardJP has suggested, and just give a few numbers that look okay for now. Since I can create another PCD map, which looks much more similar to the real-world one than the current PCD map, I would like to create the map once again and then try matching them.

[22-08-16] Update
Trying to test, but stuck at another issue (lidar driver)
I created some PCD maps and tried to do some testing before closing this issue.
For testing, I'm trying to run MORAI SIM: Drive with Autwoare to see whether localization is working fine, but I'm struggling in starting a lidar driver for Autoware.Universe

Velodyne models and Ouster models of the MORAI SIM: Drive send just like real-world sensors (Same UDP package)
I'm looking for how to run Velodyne driver and Ouster driver correctly for Autoware.Universe (FYI, for Autoware.Auto, the Velodyne driver was working fine)

https://github.com/tier4/velodyne_vls is being used in the Isuzu bus for velodyne driver.

Is it possible for a lanelet2 map for Autoware can have an arbitrary position (lat, lon) as the origin of the local map coordinate system? (NOT the origin of an MGRS system)
Yes.
You should be able to provide it as UTM origin by giving as a parameter to the loader with something similar to https://github.com/autowarefoundation/autoware.universe/blob/main/map/map_loader/config/lanelet2_map_loader.param.yaml

@xmfcx @mitsudome-r Thanks a lot for sharing!
I was following through how Autoware.Universe launches velodyne drivers, and I found this one: sample_sensor_kit_launch/launch/lidar.launch.xml
Is this the lidar configuration file when launching the Autoware using the autoware_launch/launch/autoware.launch.xml and sample_sensor_kit (the default value) is used for sensor_model argument?

authors: VRichardJP mitsudome-r sglee-morai xmfcx


text:
Autoware does not check for collision on the vehicle actual trajectory

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I'm convinced that this is not my fault but a bug.

Description
Today, I observed autoware does not want to stop for an obstacle right in front of the ego vehicle if everything happens outside the path.
I guess a screenshot is worth a thousand words (situation reproduced with the planning simulator):

R.I.P

Expected behavior
Autoware should not hit obstacles on its trajectory
Actual behavior
Better not be a human in a simulation.
Steps to reproduce
See the screenshot
Versions
No response
Possible causes
No response
Additional context
No response

You can enable obstacle_collision_checker to check collision on predicted_trajectory. It should solve your problem.

Wow! That's life saving!
I'll try it.

If I understand correctly, the node should create a diagnostic error if there is something on the predicted trajectory? Then with the right configuration, autoware should trigger emergency when this happens, correct?
I have backported the latest control.launch.py to my tree to add the obstacle_collision_checker. When I start autoware, I can see the module is running and is generating diagnostics.
I have reproduced my first screenshot scenario with a golf cart and traffic cones. However, in that situation, the diagnostic is always "OK". Even when the cone is right in front of the car (and detected as obstacle) the collision checker still generates "OK" diagnostics.

Hmm I see, I tried too but module created "ERROR" diagnostic if there is an object in braking distance. But after vehicle stops, you will see "OK" because object is not in braking distance of vehicle (because velocity of vehicle is 0).
https://user-images.githubusercontent.com/45468306/201379224-7a4f2a7b-325e-4587-8135-231810bdb6a5.mp4
Btw, it will not trigger the emergency, to do this, we should add it in diagnostic_aggregator and system_error_monitor.param.yaml to be taken into account by emergency_handler. Because it is optional, we did not add it.

If the speed is taken into account, isn't there a risk the module will say "OK" when the vehicle is stopped right in front of the obstacle? then even with the emergency handling, autoware would restart, then stop, then restart, etc?

To avoid this, you can enable emergency_holding. If there is a problem, emergency will be true until you clear the emergency.
But you are right, obstacle_collision_checker is just for emergency situations. Planning should also check the predicted_trajectory for obstacle stop. I am going to work on this if it is okey for you as well.

@brkay54 @VRichardJP There's another similar discussion, so could you check this, and continue the discussion in there if necessary?

Yes, this issue is being tracked in https://github.com/autowarefoundation/autoware/discussions/2859
Please continue there.

authors: VRichardJP brkay54 satoshi-ota xmfcx


text:
Feature: Remove Autoware limitations about map boundaries

Checklist

[X] I've read the contribution guidelines.
[X] I've searched other issues and no duplicate issues were found.
[X] I've agreed with the maintainers that I can plan this task.

Description
Provide a Dynamic Lanelet2 map loader. (Problem detail below)
Autoware Projections
Autoware has a limitation for map boundaries. Map Projection Loader modules can work for MGRS, Local Cartesian UTM and Transverse Mercator projections. MGRS Projection works inside of the 100km x 100km MGRS grids. Local Cartesian UTM works inside of a UTM zone. Transverse Mercator does not have a limitation.
Problems are detected via a lanelet2 map generated via Python script that uses CommonRoad API between Istanbul and Ankara in Turkey. The road lenght is approximately 600km.

Problems with Available Projections
MGRS
The 100km x 100km MGRS grid visualization can be found below. The road overlays more than 1 MGRS grid and the nodes and line segments reaches out the other MGRS grids than the main one keep being inside in the main MGRS grid. So, the 600km road is stuck inside of the main MGRS grid. It seems like the line segments are bouncing from the edges of the main grid. The image related to that problem can be found below.

LocalCartesianUTM
Same situation is valid for Local Cartesian UTM projection. Only difference is the map boundaries. Map boundaries here is not 100km x 100km MG

RS grids, they are the UTM zone grids. So, this projection involves much area than MGRS projection but still it is insufficient for long routes. UTM projection visualization on the same road and the same error can be seen below.

TransverseMercator
Transverse Mercator projection hasn't any boundary limitations. Additionally, we are able to load all the roads into the Autoware and visualize it in RViz2 with Planning Simulator or Logging Simulator. However, when we get away from the origin of the lanelet2 map, the line segments, TF markers, 2D Pose Estimate or 2D Goal Pose arrows start to corrupt because of the large numbers. This very looks like a floating point error. RViz2 screenshot can be seen in below GIF. Another issue with Transverse Mercator is corruptions in the map while getting away from the origin. In far places, the map loses its correctness due to the projection surface.

Additionally, due to the large numbers of coordinates and 123 MB lanelet2(.osm) file makes the system very heavy. While this GIF is recording, nearly 20 GBs of RAM is used from the computer by Autoware Planning Simulator.
Purpose
Autoware needs to be compatible with long range drivable lanelet2 maps. While doing that, RAM usage must be kept low.
Possible approaches
Dynamic Lanelet2 Map Loading
Autoware can load point cloud maps dynamicly. This provides better physical component usage for PCs and accelerates the whole Autoware due to the light system. Same situation is valid for Lanelet2 Maps. Error determination tests are implemented with 600 km road overlays different projection zones. Observed that, only Transverse Mercator project is available to use all the map at once but it has floating point problem and loses correctness while getting away from the origin.
MGRS and LocalCartesianUTM works in a boundary and they preserves their sufficient accuracy in the boundary. So, for keep getting sufficient accuracy from the map and for keeping the map outsourcing lower, Lanalet2 Maps can loaded dynamicly as the car moves. As a matter of fact, 10 or 5 km MGRS grids can be used for dynamicly loading and this will provide more compatible position data generation with the real world.
Definition of done

Autoware loads and uses Lanelet2 maps overlays large areas.
Maps does not uses too much source from the computer.
Observing no corruption while getting away from the origin.

Looks very complicated. Just a random idea:

split the lanelet map into N lanelet maps (~MGRS size), each with its own local projection (referred by frame "mapXX").
"local" lanelets maps should overlap to have smooth transition from one to the next (similar to dynamic PCD loading).
A global server tracks vehicle pose and broadcast the current "mapXX" frame to use. Since all frames are fixed and known on startup, it is not a problem if some nodes use different map frame during transition phases (as long as they carefully check and transform each data).
Obviously, it means all node currently using hardcoded "map" frame must be updated.
A new global planner must be implemented as the global path from start to goal cannot be known anymore. For this, we could preprocess each "local" lanelet map into a graph saying which lanelet id at the edge of the "local" map can reach which other lanelet id at the edge. The actual local path would only be computed once the vehicle reach the local map.

Dynamic Lanelet2 Map Loading
This concept is found necessary for using Autoware in large areas. The main idea is changing the loaded map area while the body is moving. To do that, MGRS grids can be used. Because MGRS grids enables to use the squares of 100km, 10km and 1km areas. We think that 1km MGRS grids will be ok to use. In that way, planning modules can work easily and all the pipeline would use lower amount time towards to larger areas. Here is a demonstration of the idea with a GIF below.

Editing Planning Pipeline According to Map Loading
With dynamicly loaded Lanelet2 maps, a problem is occuring. In large areas, we are not able to give a goal point with this planning implementation. Because the lanelet of the goal point could not be known with dynamicly loaded small Lanelet2 maps.
This situation can be solved with the suggestion made by @VRichardJP, which is using a graph structure to calculate the route with a full Lanelet2 map. This can work I think but this means one or several new package with various new nodes needed to be written and tested. This will cost too much time and too much energy to implement in my opinion. Additionally, I cannot guess the computational cost and how would be the performance in this stage. Still this can be the solution. Thank you for your sharings @VRichardJP.
Another solution can be using a different global planner with implementing it in the Autoware planning pipeline. This solution makes similar approximations with @xmfcx 's thoughts in another discussion about planning performence in large areas from @StepTurtle. We planned to use a global planner for large areas before giving a goal point. In this way, we can know which roads will be used in the route. Additionally, we got a planned route as a linestring and we can use it with planning pipeline.
The main idea here is taking the global planned route and giving goal points in each 1.5 or 2 kilometers to planning pipeline (Goal point can change according to the grid size will be used in Dynamic Lanelet2 Map Loading). Planning works fine in small environments has overall 1 kilometer radius. In that point of view, if we use goal points in a limited length and dynamicly changes the goal points along the globally planned route, it should work. Here is a demonstration of the general plan below.

Due to the complexity of the issue, we would like to discuss this before implement anything. Any comment or ideas would be great. @xmfcx @mitsudome-r

@xmfcx @mitsudome-r
We can use this issue for only Dynamic Lanelet2 Map Loading and open another issue for route planning if needed. Also those already exist for route planning for large areas:

#5940
Investigation on using Large Scale Lanelet2 Map in Autoware | Planning Simulator #4075

Using an external planner is not planned right now. The main aim is using mission_planner for large areas too. mission_planner can plan a route even in very long routes as can be seen in this video.
The main goal right now, using mission_planner with Dynamic Lanelet2 Loading. We are aiming it to use in MGRS projection for reaching the lowest projection corruption. However, MGRS projection has a limitation in a 100km x 100km MGRS grid. Besides that being a limitation, it can be an opportunity to use it on our good.
In large areas (over 100km routes), we are aiming to use mission_planner to plan the route. Once it plans the route, we can hold the lanelet ids inside of the route. The only issue here is to keeping coordinates low for not getting floating point errors. We thought that MGRS limitations can help with this. Here is a demonstration of what is planned.

As can be seen in the demonstration, in large areas, we change MGRS grids. In that situation, we will see that the lanelet2 maps of different MGRS grids will overlay due to the MGRS boundaries. To avoid that situation, we are planning that to use MGRS strings to find which MGRS grid the moving vehicle is in. Additonally, with Dynamic Lanelet2 Loading, it is aimed that to load lanelet2 maps according to MGRS string of moving vehicle.
Additionally, for keeping the coordinates low, we can apply the same method to point cloud maps as NDT input. With an MGRS string specification for each lanelet2 map and each point cloud will be able to solve the localization and mapping problem.
We will test this with a point cloud taken from AWSIM and a lanelet2 map generated via CommonRoad API at the crossing of 2 MGRS grids.
@xmfcx @mitsudome-r

MGRS grid cannot handle goals that is outside 100km x 100km, but can't we use other projection type for longer distance? (e.g., using TransverseMercator projection)
There could be distortion in the long distance anyways, but the topology of lane connection doesn't depend on the projection error so mission_planner should be able to do the route calculation just fine.

This issue is related to the MGRS boundaries. We can use LocalCartesian UTM or Transverse Mercator projections in Autoware. In real-life conditions, there won't be a drive that is too long. So, MGRS boundaries are not a problem right now.
We have opened PRs for Dynamic Lanelet2 Map Loading part. They are waiting for reviews.

https://github.com/autowarefoundation/autoware.universe/pull/6241
https://github.com/autowarefoundation/autoware_launch/pull/872
https://github.com/autowarefoundation/autoware_msgs/pull/81
https://github.com/autowarefoundation/autoware_common/pull/234
https://github.com/autowarefoundation/autoware.universe/pull/6393

authors: VRichardJP ataparlar mitsudome-r


text:
fix(tier4_autoware_api_launch): typo

Description

Fix arg name typo
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

It's good to link related PRs so that we can review why this typo was added. #779

source: gharchive/pull-request

authors: kenji-miyake shmpwk


text:
feat(detected_object_validator): parameterize validator threshold for each label

Description
This PR add parameter for min_pointcloud_num in obstacle_pointcloud_based_validator which is hardcoded in validator node file. For small categories this value can be set to a smaller one than the current value of 10.
Tests performed
WIP
Notes for reviewers

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[ ] I've confirmed the contribution guidelines.
[ ] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.
[ ] The PR has been properly tested.
[ ] The PR has been reviewed by the code owners.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.
[ ] The PR is ready for merge.

After all checkboxes are checked, anyone who has write access can merge the PR.

https://github.com/autowarefoundation/autoware.universe/pull/2894


text:
fix(behavior_path_planner): fix turn signal endless loop

Description

fix turn signal endless loop
https://github.com/autowarefoundation/autoware.universe/issues/5440
Tests performed

https://evaluation.tier4.jp/evaluation/reports/c40eba8d-7d8e-5aa6-9c33-071b5b7841ac?project_id=prd_jt
1665/1667
Effects on system behavior

Not applicable.
Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

@danielsanchezaran
First of all, thank you so much for the report!!!!! That's super helpful.
Could you review this PR and approve after your all comments are resolved?
And could you close the issue after merging this PR?
I will approve this PR because maintainer's approve is necessary and you are not the maintainer for now. But mergeing will be handled by @.kosuke55 san!
When close the issue, we really appreciate it if you upload the video that problem is solved.
(Just uploading the same video you shared internally

@kosuke55 CC @kyoichi-sugahara
LGTM. Thanks for your quick response. With this fix the issue has been solved!


text:
Way Planner cannot generate global path for lane change case

Bug report
Required information:

Operating system and version: OS and version:

 Ubuntu 16.04

Autoware installation type:

 From source

Autoware version or commit hash

 tag v.1.11

ROS distribution and version:

 Kinetic

ROS installation type:

 from binaries

Package or library, if applicable:

N/A

Description of the bug

We follow demo steps from youtube to test lane change of path planning.
Lane Change of Autoware (https://www.youtube.com/watch?v=C_4rXvKPWMY)

According to demo video, way_planner can plan global path for lane change case on moriyama map. We upgrade Autoware to 1.11.0, we try to follow demo video step by step to plan global path with way_planner. But way_planner cannot generate global path with below error message.
PlannerH -> Plan (A) Failed, Trying Plan (B).
Can't Generate Global Path for Start (X:-14770.6000061, Y:-84757.3999996, Z:39, A:2.35619449019
Lon:0, Lat:0, Alt:0, Dir:0) and Goal (X:-14878.6000061, Y:-84713, Z:39, A:-2.73146579822 Lon:0, Lat:0, Alt:0, Dir:0)
Can way_planner support plan path for lane change case on Autoware 1.11.0?
Steps to reproduce the bug

Launch Autoware 1.11.0 , then load vector map from moriyama map folder

Launch vel_pose_connect with "Simulation Mode"

Launch way_planner by command, then assign inital pose with '2D pose estimate' button via RVIZ

Next, assign goal pose of lane change case with '2D Navi Goal' button via RVIZ

Expected behavior

Way planner should plan global path for lane change case like demo video

Actual behavior

Way planner gets failed to plan global path with error message

Screenshots

Additional information

Thanks for using Autoware and for your question.
However the issue tracker is for posting confirmed bugs and feature requests.

Can way_planner support plan path for lane change case on Autoware 1.11.0?

This question seems to be better asked at the ROS Answers website following our support guidelines: https://github.com/autowarefoundation/autoware/wiki/Support-guidelines#ros-answers. Please pay particular attention to the information we ask you to provide.

Dear Developers,
Many thanks for your reply. I will report this issue to proper website by your suggestion.
Thanks for your help.

Hello, I'm seeing the same issue with version 1.13/melodic. Has this been fixed?

authors: SenRamakri YaoHsuan mitsudome-r

repo: autowarefoundation/autoware


text:
feat(docker): add universe-sensing-perception runtime stage

Description

[x] #5170
[x] #5171

This PR adds universe-sensing-perception runtime stage and generates the image.

Tests performed
https://github.com/youtalk/autoware/pull/104
https://github.com/youtalk/autoware/actions/runs/10715348730
Effects on system behavior

Not applicable.
Interface changes

Pre-review checklist for the PR author
The PR author must check the checkboxes below when creating the PR.

[x] I've confirmed the contribution guidelines.
[x] The PR follows the pull request guidelines.

In-review checklist for the PR reviewers
The PR reviewers must check the checkboxes below before approval.

[ ] The PR follows the pull request guidelines.

Post-review checklist for the PR author
The PR author must check the checkboxes below before merging.

[ ] There are no open discussions or they are tracked via tickets.

After all checkboxes are checked, anyone who has write access can merge the PR.

Should we test if we can run perception packages within this container? (e.g., launching perception_component.launch.xml from this container?)

How about changing some the namings:

core-devel to core -> since we don't have two seperate core images we can just use core
universe-sensing-perception to sensing-perception -> because universe and universe-devel images would be all-inclusive(monolithic) images so keeping sensing-perception as it is may provide better understanding.

Than the final names would be:
- base
- core
- universe, universe-devel (monolithic image)
- sensing-perception, sensing-perception-devel (modular image)
- planning-control, planning-control-devel (modular image)
- etc. (modular image)

@oguzkaganozt Though this PR should focus only on sensing-perception problem,

core-devel to core is not acceptable because core-devel indicates a development container. I hope a core runtime container will happen in the future.
universe- prefix is based on Autoware Universe/Core concept. I would like to maintain the prefix.
All-inclusive universe and universe-devel will be deprecated after the 3rd step completed https://github.com/orgs/autowarefoundation/discussions/4661#discussioncomment-9995806.

I can understand the universe/core concept behind the universe-perception-sensing naming.
But would core runtime image really be usefull for the end-user ? if not maybe omiting this one and keeping base and core images as is can be a simpler naming scheme.

@mitsudome-r I'm not sure which launch file is better to run but the ros2 launch autoware_pointcloud_preprocessor lanelet2_map_filter_node.launch.xml has been successfully ran.
I hope it is enough to evaluate the runtime container validity.
$ docker run --rm -it youtalk/autoware:latest-universe-sensing-perception-cuda-amd64
root@ab1cc4b4a313:/autoware# ros2 launch autoware_
autoware_ad_api_specs                         autoware_elevation_map_loader                 autoware_kalman_filter                        autoware_object_merger                        autoware_radar_fusion_to_detected_object      autoware_testing
autoware_adapi_v1_msgs                        autoware_euclidean_cluster                    autoware_lanelet2_extension                   autoware_object_range_splitter                autoware_radar_object_clustering              autoware_tracking_object_merger
autoware_adapi_version_msgs                   autoware_external_api_msgs                    autoware_lanelet2_extension_python            autoware_object_velocity_splitter             autoware_radar_object_tracker                 autoware_traffic_light_arbiter
autoware_auto_common                          autoware_geography_utils                      autoware_lidar_apollo_instance_segmentation   autoware_occupancy_grid_map_outlier_filter    autoware_radar_scan_to_pointcloud2            autoware_traffic_light_classifier
autoware_bytetrack                            autoware_gnss_poser                           autoware_lidar_centerpoint                    autoware_overlay_rviz_plugin                  autoware_radar_static_pointcloud_filter       autoware_traffic_light_fine_detector
autoware_cluster_merger                       autoware_grid_map_utils                       autoware_lidar_transfusion                    autoware_path_distance_calculator             autoware_radar_threshold_filter               autoware_traffic_light_map_based_detector
autoware_cmake                                autoware_ground_segmentation                  autoware_lint_common                          autoware_pcl_extensions                       autoware_radar_tracks_msgs_converter          autoware_traffic_light_multi_camera_fusion
autoware_common_msgs                          autoware_image_diagnostics                    autoware_livox_tag_filter                     autoware_perception_msgs                      autoware_radar_tracks_noise_filter            autoware_traffic_light_occlusion_predictor
autoware_compare_map_segmentation             autoware_image_projection_based_fusion        autoware_localization_msgs                    autoware_perception_rviz_plugin               autoware_raindrop_cluster_filter              autoware_traffic_light_visualization
autoware_control_msgs                         autoware_image_transport_decompressor         autoware_map_based_prediction                 autoware_planning_msgs                        autoware_sensing_msgs                         autoware_universe_utils
autoware_crosswalk_traffic_light_estimator    autoware_imu_corrector                        autoware_map_msgs                             autoware_point_types                          autoware_shape_estimation                     autoware_utils
autoware_detected_object_feature_remover      autoware_internal_msgs                        autoware_mission_details_overlay_rviz_plugin  autoware_pointcloud_preprocessor              autoware_simple_object_merger                 autoware_vehicle_info_utils
autoware_detected_object_validation           autoware_iv_external_api_adaptor              autoware_motion_utils                         autoware_probabilistic_occupancy_grid_map     autoware_system_msgs                          autoware_vehicle_msgs
autoware_detection_by_tracker                 autoware_iv_internal_api_adaptor              autoware_multi_object_tracker                 autoware_radar_crossing_objects_noise_filter  autoware_test_utils
root@ab1cc4b4a313:/autoware# ros2 launch autoware_pointcloud_preprocessor lanelet2_map_filter_node.launch.xml
[INFO] [launch]: All log files can be found below /root/.ros/log/2024-09-06-04-49-56-443651-ab1cc4b4a313-124
[INFO] [launch]: Default logging verbosity is set to INFO
[INFO] [vector_map_filter_node-1]: process started with pid [125]

authors: mitsudome-r oguzkaganozt youtalk


text:
Add ConcurrentRepolls to make the polling throughput configurable

This adds a ConcurrentRepolls variable to the Snowball Parameters and issues a repoll() call in both Snowman and Avalanche when the number of pending polls is below ConcurrentRepools.
Addresses https://github.com/ava-labs/gecko/issues/25

@StephenButtolph Thank you for the review, I'll update with the changes you proposed above.

@StephenButtolph I've updated the PR with the changes that you outlined. Still working on understanding some of the basics of the system and very much open to any more pointers and ways to improve this. Thanks for the reviews so far.

repo: ava-labs/gecko


text:
🛑 Email (Google) is down

In e874d05, Email (Google) (https://mail.google.com) was down:

HTTP code: 0
Response time: 0 ms

Resolved: Email (Google) is back up in 74e6f39.

repo: avcga/uptime


text:
malloc error and crash

julia> using MiniFB

julia> include(joinpath(pathof(MiniFB),"..","..","example","Plasma.jl"))
Window created using X11 API

# closed the window with esc after a couple seconds

julia> macro m()
       end
malloc(): mismatching next->prev_size (unsorted)

signal (6): Aborted
in expression starting at none:0

I ran into this and had to pkill julia:
julia> plasma()
Window created using X11 API

julia> malloc(): mismatching next->prev_size (unsorted)

signal (6): Aborted
in expression starting at none:0

^C

^C^C^C^C
signal (15): Terminated
in expression starting at none:0
__lll_lock_wait_private at /usr/lib/libc.so.6 (unknown line)
__libc_calloc at /usr/lib/libc.so.6 (unknown line)
jl_getFunctionInfo at /buildworker/worker/package_linux64/build/src/debuginfo.cpp:1230
jl_print_native_codeloc at /buildworker/worker/package_linux64/build/src/stackwalk.c:652
jl_critical_error at /buildworker/worker/package_linux64/build/src/signal-handling.c:239 [inlined]
sigdie_handler at /buildworker/worker/package_linux64/build/src/signals-unix.c:204
_IO_funlockfile at /usr/lib/libpthread.so.0 (unknown line)
gsignal at /usr/lib/libc.so.6 (unknown line)
abort at /usr/lib/libc.so.6 (unknown line)
__libc_message at /usr/lib/libc.so.6 (unknown line)
malloc_printerr at /usr/lib/libc.so.6 (unknown line)
_int_malloc at /usr/lib/libc.so.6 (unknown line)
__libc_malloc at /usr/lib/libc.so.6 (unknown line)
operator new at /src/gcc-7.3.0/libstdc++-v3/libsupc++/new_op.cc:50
allocate at /usr/local/include/c++/7.3.0/ext/new_allocator.h:111 [inlined]
allocate at /usr/local/include/c++/7.3.0/bits/alloc_traits.h:436 [inlined]
_M_get_node at /usr/local/include/c++/7.3.0/bits/stl_tree.h:588 [inlined]
_M_create_node<int const&> at /usr/local/include/c++/7.3.0/bits/stl_tree.h:642 [inlined]
operator()<int const&> at /usr/local/include/c++/7.3.0/bits/stl_tree.h:556 [inlined]
_M_insert_<int const&, std::_Rb_tree<int, int, std::_Identity<int>, std::less<int>, std::allocator<int> >::_Alloc_node> at /usr/local/include/c++/7.3.0/bits/stl_tree.h:1753 [inlined]
_M_insert_unique<int const&> at /usr/local/include/c++/7.3.0/bits/stl_tree.h:2096
insert at /usr/local/include/c++/7.3.0/bits/stl_set.h:501 [inlined]
emit_function at /buildworker/worker/package_linux64/build/src/codegen.cpp:5848
jl_emit_code at /buildworker/worker/package_linux64/build/src/codegen.cpp:6431
jl_emit_codeinst at /buildworker/worker/package_linux64/build/src/codegen.cpp:6465
_jl_compile_codeinst at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:97
jl_generate_fptr at /buildworker/worker/package_linux64/build/src/jitlayers.cpp:302
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1964
jl_compile_method_internal at /buildworker/worker/package_linux64/build/src/gf.c:1919 [inlined]
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2224 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
prompt! at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/LineEdit.jl:2447
run_interface at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/LineEdit.jl:2350
jfptr_run_interface_70362.clone_1 at /home/miguel/bin/julia-1.5.1/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
run_frontend at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/REPL/src/REPL.jl:1144
#38 at ./task.jl:356
jfptr_YY.38_71508.clone_1 at /home/miguel/bin/julia-1.5.1/lib/julia/sys.so (unknown line)
_jl_invoke at /buildworker/worker/package_linux64/build/src/gf.c:2214 [inlined]
jl_apply_generic at /buildworker/worker/package_linux64/build/src/gf.c:2398
jl_apply at /buildworker/worker/package_linux64/build/src/julia.h:1690 [inlined]
start_task at /buildworker/worker/package_linux64/build/src/task.c:707
unknown function (ip: (nil))
unknown function (ip: (nil))
Allocations: 11500038 (Pool: 11495503; Big: 4535); GC: 12

I might need some help debugging this. I can't replicate this on mac or windows, and I haven't been able to access a linux machine with a GUI terminal (and attempts to set up VNC on a few machines that I do have access to haven't yet suceeded 😞 ) . Might help to create an rr trace?

I'm having trouble running a trace. It seems that Julia crashes so bad that the trace is not created before Julia terminates. I'm getting this stacktrace. Notes: I got the malloc() error immediately after hitting macro m()<enter>. Then Julia was stuck after the signal (6). The stacktrace appeared after I hit Ctrl-C. Any suggestion on how to get a trace under these conditions? I'll be happy to run any tests you need.

$ julia --bug-report=rr
[ Info: Loading BugReporting package...
rr: Saving execution to trace directory `/home/miguel/.julia/artifacts/jl_x1ZhGZ/julia-0'.
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.1 (2020-08-25)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> using MiniFB

julia> include(joinpath(pathof(MiniFB),"..","..","example","Plasma.jl"))
Window created using X11 API

julia> macro m()malloc(): mismatching next->prev_size (unsorted)

signal (6): Aborted
in expression starting at none:0

^CERROR: InterruptException:
Stacktrace:
 [1] poptask(::Base.InvasiveLinkedListSynchronized{Task}) at ./task.jl:704
 [2] wait at ./task.jl:712 [inlined]
 [3] wait(::Base.GenericCondition{Base.Threads.SpinLock}) at ./condition.jl:106
 [4] wait(::Base.Process) at ./process.jl:622
 [5] success at ./process.jl:483 [inlined]
 [6] run(::Cmd; wait::Bool) at ./process.jl:440
 [7] run at ./process.jl:438 [inlined]
 [8] (::BugReporting.var"#4#5"{String,Tuple{Cmd,Array{Any,1}},Array{SubString{String},1}})(::String) at /home/miguel/.julia/packages/BugReporting/DYZaI/src/BugReporting.jl:84
 [9] #2 at /home/miguel/.julia/packages/JLLWrappers/m1BKg/src/runtime.jl:47 [inlined]
 [10] withenv(::JLLWrappers.var"#2#3"{BugReporting.var"#4#5"{String,Tuple{Cmd,Array{Any,1}},Array{SubString{String},1}},String}, ::Pair{String,String}, ::Vararg{Pair{String,String},N} where N) at ./env.jl:161
 [11] withenv_executable_wrapper(::BugReporting.var"#4#5"{String,Tuple{Cmd,Array{Any,1}},Array{SubString{String},1}}, ::String, ::String, ::String, ::Bool, ::Bool) at /home/miguel/.julia/packages/JLLWrappers/m1BKg/src/runtime.jl:46
 [12] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [13] invokelatest at ./essentials.jl:709 [inlined]
 [14] #rr#7 at /home/miguel/.julia/packages/JLLWrappers/m1BKg/src/products/executable_generators.jl:7 [inlined]
 [15] rr at /home/miguel/.julia/packages/JLLWrappers/m1BKg/src/products/executable_generators.jl:7 [inlined]
 [16] rr_record(::Cmd, ::Vararg{Any,N} where N; trace_dir::String) at /home/miguel/.julia/packages/BugReporting/DYZaI/src/BugReporting.jl:74
 [17] (::BugReporting.var"#15#16"{Array{Any,1},Cmd})(::String) at /home/miguel/.julia/packages/BugReporting/DYZaI/src/BugReporting.jl:130
 [18] create_artifact(::BugReporting.var"#15#16"{Array{Any,1},Cmd}) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/Pkg/src/Artifacts.jl:214
 [19] make_interactive_report(::String, ::Array{Any,1}) at /home/miguel/.julia/packages/BugReporting/DYZaI/src/BugReporting.jl:129
 [20] make_interactive_report(::String) at /home/miguel/.julia/packages/BugReporting/DYZaI/src/BugReporting.jl:123
 [21] #invokelatest#1 at ./essentials.jl:710 [inlined]
 [22] invokelatest at ./essentials.jl:709 [inlined]
 [23] report_bug(::String) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/InteractiveUtils/src/InteractiveUtils.jl:379
 [24] exec_options(::Base.JLOptions) at ./client.jl:244
 [25] _start() at ./client.jl:506
0[~] $

authors: JeffBezanson aviks mbaz


text:
Customer Segmentation Engine for Mall Customers

Description
Customer Segmentation Engine for Mall Customers
Fixes #1959

Checklist:

[x] My code follows the style guidelines(Clean Code) of this project
[x] I have performed a self-review of my own code
[x] I have commented my code, particularly in hard-to-understand areas
[x] I have created a helpful and easy to understand README.md
[x] My documentation follows Template for README.md

Project Metadata
If there is no-file/nothing to fill the below fields with, then type: none
Example: If no requirements.txt needed/present, then type none in Requirments.
Category:

[x] AI/ML

Title: Customer Segmentation Engine for Mall Customers
Folder: Customer Segmentation
Script: Customer Segmentation/Customer Segmentation Engine for Mall Customers.ipynb
Contributor: ayush-09
Description: Customer Segmentation Engine for Mall Customers

@avinashkranjan Please review this PR

repo: avinashkranjan/Amazing-Python-Scripts


text:
Add Ashtadhyayi Sutrapatha

Text is available from ashtadhyayi.com/ (I can push text files in)
One source for audio is http://surasa.net/music/samskrta-vani/ashtadhyayi.php
Another is https://www.sanskritfromhome.in/course/ashtadhyayi/ (While this one is not freely available, it is very precise and devoid of some occasional errors, thus more useful for someone trying to learn to chant/memorise)
I tried aligning both versions. I am not sure if it's my system or it's about the nature of the text, but the alignments I obtained are pretty bad.

Pushed the ashtadhyayi alignment as well. Managed to figure out why the alignment was poor, culprit being improper installation of aeneas.
Short version is,

Install using pip install aeneas, not pip install py3-aeneas even if latter seems like an updated fork.
Don't forget to apt-install libespeak-dev (on top of ffprobe, ffmpeg, espeak)

Thanks. I should have added these notes to the README (at least installing aeneas and the other libs). Will do that, set up a GH action and merge this PR.

Closed via PR #4

authors: avinashvarna hrishikeshrt


text:
Deployment of ramayana alignment

What does that mean, @avinashvarna please tell ?
See my http://samskrtam.ru/parallel-corpus/

Thanks for sharing the link - this is similar to valmikiramayan.net in Russian. Will be useful for training multi-lingual models.
This repo though is for a demo of aligning audio with text to create a "read along" mode. I was planning to send a note in the Sanskrit programmers mailing list later this week with more details, but you can take a sneak peek here - https://avinashvarna.github.io/rAmAyaNa-paThanam/
If you play the audio at the top of the page, the corresponding word being uttered should be highlighted (with some occasional lag/errors). Scrolling to a different point in the audio should cause the text to also advance to the corresponding section.

Hope that was clear. Closing this issue.

authors: avinashvarna gasyoun

repo: avinashvarna/rAmAyaNa-paThanam


text:
'MMDetObjectDetector3D' object has no attribute 'save_folder'

Hello
I was trying the tracking example in the documentation and stuck with an error.
While running the below code snippet, I am getting an error
# %%capture
# ^^ suppress output
percep_res_frames, percep_res_seq, _ = avapi.evaluation.get_percep_results_from_folder(
    DM, P.save_folder, sensor_eval='main_lidar', multiprocess=True)
track_res_frames, track_res_seq, track_res_exp = avapi.evaluation.get_track_results_from_folder(
    DM, T.save_folder, sensor_eval='main_lidar', multiprocess=False)

The error came as:
---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
Cell In[4], line 4
      1 # %%capture
      2 # ^^ suppress output
      3 percep_res_frames, percep_res_seq, _ = avapi.evaluation.get_percep_results_from_folder(
----> 4     DM, P.save_folder, sensor_eval='main_lidar', multiprocess=True)
      5 track_res_frames, track_res_seq, track_res_exp = avapi.evaluation.get_track_results_from_folder(
      6     DM, T.save_folder, sensor_eval='main_lidar', multiprocess=False)

AttributeError: 'MMDetObjectDetector3D' object has no attribute 'save_folder'

I checked the MMDetObjectDetector3D class and its parent classes and can't able to find out the usage of the save_folder.
Do we need to create the folder our self? What data will be saved in that folder?
Thanks for the feedback

Thanks for pointing this out. I'm in the middle of a big update to the code
base and I'll revisit the tutorials once that is completed. Hopefully this
weekend.
On Wed, Feb 7, 2024 at 8:00 AM Rony Shaji @.***> wrote:

Hello
I was trying the tracking example in the documentation and stuck with an
error.
While running the below code snippet, I am getting an error
%%capture
^^ suppress output
percep_res_frames, percep_res_seq, _ = avapi.evaluation.get_percep_results_from_folder(
DM, P.save_folder, sensor_eval='main_lidar', multiprocess=True)
track_res_frames, track_res_seq, track_res_exp = avapi.evaluation.get_track_results_from_folder(
DM, T.save_folder, sensor_eval='main_lidar', multiprocess=False)
The error came as:

AttributeError                            Traceback (most recent call last)
Cell In[4], line 4
1 # %%capture
2 # ^^ suppress output
3 percep_res_frames, percep_res_seq, _ = avapi.evaluation.get_percep_results_from_folder(
----> 4     DM, P.save_folder, sensor_eval='main_lidar', multiprocess=True)
5 track_res_frames, track_res_seq, track_res_exp = avapi.evaluation.get_track_results_from_folder(
6     DM, T.save_folder, sensor_eval='main_lidar', multiprocess=False)
AttributeError: 'MMDetObjectDetector3D' object has no attribute 'save_folder'
I checked the MMDetObjectDetector3D class and its parent classes and can't
able to find out the usage of the save_folder.
Do we need to create the folder our self? What data will be saved in that
folder?
Thanks for the feedback
—
Reply to this email directly, view it on GitHub
https://github.com/avstack-lab/lib-avstack-core/issues/30, or
unsubscribe
https://github.com/notifications/unsubscribe-auth/AE326ZR66W4DNV4ZGELGH2TYSN3IBAVCNFSM6AAAAABC5ZENQGVHI2DSMVQWIX3LMV43ASLTON2WKOZSGEZDEOJYGU4TOMQ
.
You are receiving this because you are subscribed to this thread.Message
ID: @.***>

Great. Thanks for the update.

Hello @roshambo919, I hope you're doing well. I was wondering if there have been any developments regarding the issue mentioned earlier?

So sorry for the delay. I just now merged in some updates to the main branch. The biggest changes are the incorporation of config-based building of modules. In general, looking at the tests folder is the best way to see AVstack working in action. The tests are required to pass before I merge anything in, so they should always work if things are set up properly. The documentation online is out of date - I will have a student work on fixing that.
I'm targeting an official release of a v0.1 AVstack on pypi at around April 1, FYI.
In the meantime, if you want support for a particular use case of AVstack, feel free to point me to your git repository and I can provide some support.

@roshambo919 Thanks for the feedback. I have updated the sub-modules based on your change, but unfortunately I was not able to rectify the error. The error says that the below object 'P' does not have an attribute which used in the next stage (evaluation)
P = avstack.modules.perception.object3d.MMDetObjectDetector3D(
        model='pointpillars', dataset='kitti', threshold=0.7,
        save_output=True, save_folder=result_folder)

Here the folder which the results should be saves is passed as an argument, which is an attribute of P. But unfortunately P does not have any attribute like save_folder (P.save_folder).
percep_res_frames, percep_res_seq, _ = avapi.evaluation.get_percep_results_from_folder(
    DM, P.save_folder, sensor_eval='main_lidar', multiprocess=True)

I checked the MMDetObjectDetector3D class and can't find anything related to save_folder or something. So there is no folder available and the evaluation can't be performed. I am still new to the AVstack and primarily trying out based on the tutorial available on the website.
Thanks
Rony Shaji

Yes, I removed the save_folder input argument completely. Your code should now just be:
P = avstack.modules.perception.object3d.MMDetObjectDetector3D(
        model='pointpillars', dataset='kitti', threshold=0.7
)

If you need to save the output, consider saving it manually for now or using a logging hook as in https://github.com/avstack-lab/lib-avstack-core/blob/b9d16713b35cf8f2f1cbe78924025b72ad578767/tests/utils/test_logger.py#L50
I'll try to get a logging example working again soon.

Hey @roshambo919
Hope you are doing fine. I am still trying out the AVstack and wondering whether the v0.1 AVstack is already out. As mentioned I faced some issues with the current version, and hopefully it will be solved with the new version.
Also I created a custom map of a test field in CARLA and is planning to do multi object tracking using infrastructural sensor(LIDAR) and would like to know whether I can incorporate AVstack there? I saw the tutorial to Generate Collaborative CARLA Dataset, but the methods are not much clear. So your feedback will be greatly appreciated.

Hi @ronyshaji - thanks so much for following up. There's a lot of progress going on with the carla integration including the development of lib-avstack-carla and the in-progress integration with the carla-sandbox.
Once I finish up a project this week, I'll write a tutorial for how to use it. I think you'll find it very useful.

@roshambo919 Thanks for the feedback. Hope the tutorials will cover infrastructure sensor based tracking too 🙂

@ronyshaji I patched the docs through the tracking example in a branch named "update". This includes basic perception and tracking examples. Infrastructure examples to come in the future.
https://github.com/avstack-lab/avstack-docs/tree/update

Hey @roshambo919
Thanks for the update.
Unfortunately I am still facing some issues. First of all, I think the CARLA dataset download link in the .sh is not working.
I tried to download manually still it is not working https://download_carla_datasets.sh/. KITTI and nuScenes are working as intended. Since my main concentration is CARLA, it would be great if you provide a working link to download it.
Also got an error with line 6 in base.py, I had to replace
lidar_cmap = plt.cm.get_cmap("hsv", 256)
by
lidar_cmap = plt.get_cmap("hsv", 256)
Regards
Rony Shaji

repo: avstack-lab/lib-avstack-core


text:
I tried to install, I got this.. And after modification refresh it didn't worked for me...

Class Aweb\Nexus\Support\HigherOrderTapProxy located in C:/xampp/htdocs/ocl/system/library/aweb/nexus\Support\Traits\HigherOrderTapProxy.php does not comply with psr-4 autoloading standard. Skipping.

Make sure you installed composer r3quired packages or use the complete package from the release which already contains the vendor folder.
Change de php version to 7.3. For higher versions we did not done all the tests yes

authors: awebdesign hisatnam

repo: awebdesign/nexus


text:
Date picker popup not showing up correctly when screen size is small (on mobile)

Date picker pop up looks very weird when table is narrow (like on mobile devices).
Please see screenshot attached.
Any idea how to solve this issue ?
Thanks

Can you provide a plunker?

Popups exhibit this kind of problems. Does this component provide an attachToBody configuration? It might solve this issue.

tried datepicker-append-to-body=true  - seems like it's not really affecting something.
Will try to have a plunker tomorrow

I have something more strange that the columns titles are showing in the popup datetime picker. Please suggest.

This happens because the datepicker is a nested table, although the result is inconvenient.
Can you verify if this could solve your problem, if not already applied:

Popups often exhibit this kind of problem. Does this component provide an attachToBody configuration? It might solve this issue.

Which datapicker component are you using?

I'm using this https://github.com/dalelotts/angular-bootstrap-datetimepicker
What we need is a popup datetime picker, not embedded in body. Is it because of injecting all the "td" with the title/cell recursively? I saw you have a change "tables without the directive are ignored". Is this done with the intention to solve this issue? You are doing a great work here and I really appreciate if such issue can be solved. Many thanks.

The other commit handles a similar issue, but on a top-level table element. It doesn't handle this issue because the datepicker is a table nested inside a responsive table. I'll take a closer look at this.

Thanks.

@andycwwai @ransalman Please, if possible, update to v0.4.0 and verify if issue is indeed resolved. Please re-open if necessary.

The datetime picker calendar can be shown correctly but the text and title are overlapped.

Can you provide a plunker?

Actually I believe this is another issue not related to the date picker anymore, but emerged with the previous fix. To give more context, I made selectors more strict. This could be either a syntactical error on your markup or I missed a valid markup structure.

I found out that it is css of bootstrap's theme that causes the problem: it has a style "padding:8px" on ".table>tbody>tr>td". I've tried to change your css from ".responsive > tbody > tr > td" to ".table.responsive > tbody > tr > td" and it works. See if you have any other better idea to solve it. Once again, thank you for your great effort.

Thanks for reporting. It is really a specificity problem, previously I had a dirty hack to deal with it. It is odd that on the examples page the problem doesn't show up. And I have a unit test that asserts that the padding: 8px is overriden with 50%. Which version of bootstrap are you running on?
I can't use the class name .table, but I can repeat it like .responsive.responsive. CSS rules...

Thanks. FYI, we are using a theme "Spacelab" getting from http://bootswatch.com

Checkout v0.4.1. Hope its fixed for good. If case not, please file a new issue. :)

It works perfect! Thanks.

authors: andycwwai awerlang ransalman

repo: awerlang/angular-responsive-tables

license: mit

license_source: bigquery


text:
Radar's not working in Steam deck with xivlauncher

Everything looks good.
The auto rotation is working as expected.
However, in the windows, it shows: cur: inactive, for all bosses
I am using English version and playing in U.S server

What boss are you trying it on?
I'll need a log to debug this - see main page for details on how to grab one.

Outdated.

authors: awgil gaoyike


text:
[QUESTION] can I update the CLI without making changes to the project?

** Please describe which feature you have a question about? **
This is probably a dumb question, but I just want to update the CLI so I can use Lambda Layers and dont want to cause issues in my current project. Thx

Hello @contractorwolf
Generally the CLI handles migrations when upgrading between versions. For any migrations which require external manual intervention please review our docs here: https://docs.amplify.aws/cli/migration/lambda-node-version-update

authors: SwaySway contractorwolf

repo: aws-amplify/amplify-cli


text:
Mock in Container - Error

When I try to start the API for testing in docker container, I get the following error:
Failed to start API Mock endpoint UnknownEndpoint: Inaccessible host: localhost. This service may not be available in the us-fake-1' region.

Hi @aexei what do you mean when you say "testing in docker container"? The CLI does not use docker for mock. Can you provide more details around what resources you have in your project and detailed repro steps that led to this error?

Hi @edwardfoyle,
For our CI/CD process, we use Gitlab Pipelines and have implemented various tests (e2e, intergration, ...). For this I want to mock the API within a Gitlab CI/CD job and then run the tests against the API. Gitlab uses one Docker container per job. Initializing Amplify and getting it ready works without problem, but when I run amplify mock I get the error.
I know that in Gitlab localhost is blocked, so I created a custom Docker container. This is then executed inside the job. Unfortunately, nothing changes.  If I run the container on my Mac, everything runs without a problem.
Gitlab Job:
integration:
  stage: test
  services:
    - docker:19.03-dind
  image: docker:latest
  script:
    - docker run
        --rm
        -v ${PWD}:/data
        --env AWS_ACCESS_KEY_ID=$AWS_ACCESS_KEY_ID
        --env AWS_SECRET_ACCESS_KEY=$AWS_SECRET_ACCESS_KEY
        --env AWS_DEFAULT_REGION=$AWS_DEFAULT_REGION
        notionug/docker-aws-amplify
        sh /data/.scripts/ci/runTestIntegration.sh

Dockerfile.txt
init_project_ci.txt
runTestIntegration.txt

@edwardfoyle
I have found a solution. Apparently one part (I suspect DynamoDB) does not work on Alpine Linux. I have adjusted my DOCKERFILE to amazon linux 2 and everything works

authors: aexei edwardfoyle


text:
amplify push api not working

Before opening, please confirm:

[X] I have installed the latest version of the Amplify CLI (see above), and confirmed that the issue still persists.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting bug reports.
[X] I have done my best to include a minimal, self-contained set of instructions for consistently reproducing the issue.

How did you install the Amplify CLI?
npm
If applicable, what version of Node.js are you using?
16.13.1
Amplify CLI Version
7.6.15
What operating system are you using?
Mac
Did you make any manual changes to the cloud resources managed by Amplify? Please describe the changes made.
No manual changes made
Amplify Categories
api
Amplify Commands
push
Describe the bug
I have created multiple .graphql files in schema directory, and removed schema.graphql file from /amplify/backend/api/<API_NAME>/schema/ directory.
running amplify api gql-compile command works and returns successfully
running amplify push prompt with following message:
✔ This version of Amplify CLI introduces additional security enhancements for your GraphQL API. @auth authorization rules applied on primary keys and indexes are scoped down further. The changes are applied automatically with this deployment. This change won't impact your client code. Continue (Y/n) · yes
Choosing yes, creates an empty schema.graphql file in directory /amplify/backend/api/<API_NAME>/
As schema.graphql file is empty, this leads to following error when pushing changes
✖ There was an error pulling the backend environment dev. 🛑 An error occurred during the push operation: Syntax Error: Unexpected <EOF>
Expected behavior
In case, .graphql files placed in /amplify/backend/api/<API_NAME>/schema/ directory, those files should be used, instead of creating an empty schema.graphql in /amplify/backend/api/<API_NAME>/
Additionally, schema path specified as gqlSchemaPath in cli-inputs.json file should be used
Reproduction steps
Create .graphql files in /amplify/backend/api/<API_NAME>/schema/ directory
Remove schema.graphql file in /amplify/backend/api/<API_NAME>/ directory
Run amplify push
On following prompt, choose yes
✔ This version of Amplify CLI introduces additional security enhancements for your GraphQL API. @auth authorization rules applied on primary keys and indexes are scoped down further. The changes are applied automatically with this deployment. This change won't impact your client code. Continue (Y/n) · yes
Error is thrown 🛑 An error occurred during the push operation: Syntax Error: Unexpected <EOF>
GraphQL schema(s)

# Put schemas below this line

Log output

# Put your logs below this line
% amplify push --y

✔ This version of Amplify CLI introduces additional security enhancements for your GraphQL API. @auth authorization rules applied on primary keys and indexes are scoped down further. The changes are applied automatically with this deployment. This change won't impact your client code. Continue (Y/n) · yes

✖ There was an error pulling the backend environment dev.
🛑 An error occurred during the push operation: Syntax Error: Unexpected <EOF>

GraphQL request:1:2
1 |
  |  ^

Additional information
No response

Hey @adilusman51 :wave: thanks for raising this! This appears to be a duplicate of https://github.com/aws-amplify/amplify-cli/issues/9624 and should be fixed with https://github.com/aws-amplify/amplify-cli/pull/9670. From the issue linked,

By deleting the empty file before answering the second question I was able to get the push to complete successfully.

Marking as a duplicate bug and closing in favor of tracking the existing item. Please subscribe to #9624 for additional updates

@josefaidt Thanks for providing an update, this is in fact a duplicate issue of #9624


text:
fix: aws-exports to be read regardless of JS project type

Description of changes
Removes ESM default export -> CJS module.exports string replace in favor of transpiling aws-exports.js with Babel, creating a module from the transpiled result, then accessing exports directly.
closed https://github.com/aws-amplify/amplify-cli/pull/10884 in favor of this approach

Issue #, if available

closes https://github.com/aws-amplify/amplify-cli/issues/5691
closes https://github.com/aws-amplify/amplify-cli/issues/10101
Description of how you validated changes

amplify-dev binary
pnpm create vite my-vite-project --template react-ts
amplify-dev init -y
amplify-dev env checkout dev   # triggers CLI to read aws-exports file

running the packaged binary directly
pnpm create vite my-vite-project --template react-ts
/absolute/path/to/amplify-pkg-macos-x64 init -y
/absolute/path/to/amplify-pkg-macos-x64 env checkout dev # triggers CLI to read aws-exports file

added 4 tests

reads ESM
reads CJS
fails if unable to parse
fails if exports is empty/null (does this have unintended consequences? can be removed)

Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #10918 (c6d68a6) into dev (1401175) will increase coverage by 0.00%.
The diff coverage is n/a.

@@           Coverage Diff           @@
##              dev   #10918   +/-   ##
=======================================
  Coverage   47.26%   47.26%
=======================================
  Files         674      674
  Lines       33211    33211
  Branches     6698     6698
=======================================
+ Hits        15696    15697    +1
+ Misses      15828    15827    -1
  Partials     1687     1687

Impacted Files
Coverage Δ

...li/src/domain/amplify-usageData/getUsageDataUrl.ts
100.00% <0.00%> (+12.50%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

open to feedback: is it worth the effort to add a comment block above this modified logic to explain why this is necessary or lean on git blame to point us back to this PR (and subsequently the closed PR with a dynamic import attempt and more info)

@josefaidt just started the test suite again, once it passes this should be good to go (and I'll approve).


text:
fix: stop timer on early push resources exit

Description of changes

prevents leaving the usage timer started on early exit from push resources
note: issue was discovered while testing notifications migration which triggers push for auth resource followed up by a push for analytics, if auth was previously pushed, first push will exit early leaving the timer started and followup push for analytics will throw an exception attempting to start a started timer

Description of how you validated changes

manual testing
unit test added

Checklist

[x] PR description included
[x] yarn test passes
[x] Tests are changed or added
[ ] Relevant documentation is changed or added (and PR referenced)
[ ] New AWS SDK calls or CloudFormation actions have been added to relevant test and service IAM policies
[ ] Pull request labels are added

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Codecov Report

Merging #11287 (79659a1) into dev (3939d41) will decrease coverage by 0.05%.
The diff coverage is 100.00%.

@@            Coverage Diff             @@
##              dev   #11287      +/-   ##
==========================================
- Coverage   49.60%   49.54%   -0.06%
==========================================
  Files         683      685       +2
  Lines       32840    32961     +121
  Branches     6700     6727      +27
==========================================
+ Hits        16290    16331      +41
- Misses      15070    15137      +67
- Partials     1480     1493      +13

Impacted Files
Coverage Δ

...i/src/extensions/amplify-helpers/push-resources.ts
36.58% <100.00%> (ø)

.../src/extensions/amplify-helpers/apply-auth-mode.ts
25.64% <0.00%> (ø)

...li/src/domain/amplify-usageData/getUsageDataUrl.ts
100.00% <0.00%> (+12.50%)
:arrow_up:

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

authors: codecov-commenter lazpavel


text:
Domain activation is Failed - [BRIEF DESCRIPTION]

App Id: d2f9x3ntwi2kis
Region: us-west-2
Step: Domain activation
Status: Failed

Note: Do not include information that is sensitive in nature such as your domain name, company etc.

Issue/question
A clear and concise description of what the issue/question is.
Error message
If there is an error message, please include it here.
Note: Be sure to check the message for sensitive information.
Additional information
Please add any other relevant information. Please feel free to include screenshots.

You don't have much context here but I might have struggled with the same issue. Does https://github.com/aws-amplify/amplify-console/issues/512#issuecomment-639892355 help?

Resolving due to age - re-open if you are still facing the issue!

authors: OzzieOrca ganipcanot hussainMansoor876

repo: aws-amplify/amplify-console


text:
amplify push vs git push

Before opening, please confirm:

[X] I have checked to see if my question is addressed in the FAQ.
[X] I have searched for duplicate or closed issues.
[X] I have read the guide for submitting questions.
[X] I have removed any sensitive information from my code snippets and submission.

App Id
d2kf2ltunv8xfd
AWS Region
eu-west-1
Amplify Hosting feature
Backend builds
Question
I am using AWS CodeCommit as git repo for my Amplify app.
What is the relation between "amplify push" and "git push"? When should I one and when the other?
The "backend" folder is part of the git repo, so when I change something in the backend code should I use "git push" or "amplify push"?
Should I use "amplify push" only after some "amplify add/remove" operations? and "git push" after other changes?
Should I use both of them in sequence?

Hi @mat-b 👋 , thanks for raising this!
Based on the information provided, I understand that you are trying to perform full stack deployments with Amplify Hosting. Since your amplify folder (backend) is part of the git repository, the best practice would be to use git push after modifying the backend code. This should then trigger the CI/CD pipeline in the Amplify Console and a new build will start.
Similarly, if you wish to add/remove categories to the existing backend, you can do so using the Amplify CLI via amplify add/remove category. To continue with full stack deployments, you will then need to use git push to send your local changes to the remote git repository instead of performing amplify push using the Amplify CLI.

repo: aws-amplify/amplify-hosting


text:
Xcode throws compile error if the data model contains a 1:1 relationship

Describe the bug
If you use AdminUI and create a 1:1 relationship between 2 models, when you generate models on the client using the CLI and try to compile for iOS using Xcode, it will not compile successfully. (I'm assuming this is not necessarily related to AdminUI but that is how I created the model - I'm assuming if it was defined in the schema.graphql it would have the same result but I have not tested it - I only used AdminUI).
To Reproduce
Steps to reproduce the behavior:

Create a new AdminUI backend.
Create 2 models (Model A and ModelB)
Create a 1:1 relationship between the 2 models
Use CLI to pull down backend
Create XCode project
Perform Pod Init, Pod install --repo-update
amplify codegen add
amplify codegen
Add generate model to xcode project
Compile Xcode

Expected behavior
Should compile successfully.
Screenshots

Environment(please complete the following information):

Amplify Framework Version: 4.41.2
Dependency Manager: [Cocoapods]
Swift Version :  version 5.3.2 (swiftlang-1200.0.45 clang-1200.0.32.28) Target: x86_64-apple-darwin20.2.0
Xcode 12.3 (12C33)
Simulator: happens on any, i was trying iPhone 12 Pro Max (if I remove the models the project compiles)

Hi, @treystudio3a
Thanks for reaching out.
My suggestion is to use lower case for the field name (use modelB instead of ModelB) which removes the complier warning.  I guess Xcode is not able to resolve this kind of confusion(having a field called ModelB and having a class called ModelB)

Hi, @treystudio3a
Thanks for reaching out.
My suggestion is to use lower case for the field name (use modelB instead of ModelB) which removes the complier warning.  I guess Xcode is not able to resolve this kind of confusion(having a field called ModelB and having a class called ModelB)

That solved my problem thank you very much

That solved my problem thank you very much

You are welcome, feel free to reach out to use for other issues.
Closing this issue

You are welcome, feel free to reach out to use for other issues.
Closing this issue

repo: aws-amplify/amplify-ios


text:
Unable to use combine framework for deleting user

Describe the bug
When I tried to use the below code as Amplify Docs described,
func deleteUser() -> AnyCancellable {
   Amplify.Auth.deleteUser()
        .resultPublisher
        .sink {
            if case let .failure(authError) = $0 {
                print("Delete user failed with error \(authError)")
            }
        }
        receiveValue: {
            print("Successfully deleted user")
        }
}

the following error occurs. No exact matches about property 'resultPublisher'.
So I looked into a file named 'AmplifyOperation+AuthPublishers' which I jumped from using the below code .Amplify.Auth.signOut(options: .init(globalSignOut: true)).resultPublisher.
    func signOutGlobally() -> AnyPublisher<Void, AuthError> {
        let resultPublisher = Amplify.Auth.signOut(options: .init(globalSignOut: true))
            .resultPublisher
        return resultPublisher
    }

And I found out that there was no AmplifyOperation extension for deleting user in  'AmplifyOperation+AuthPublishers' whereas AuthSignOutOperation, AuthSignUpOperation, and other AmplifyOperation extensions were defined. I guess this is the reason why I can't use Amplify.Auth.deleteUser().resultPublisher.

Thanks for bringing this to our attention!  We'll create a fix for this and provide an update here when it's available.

This issue has been fixed and will be included in the next release.

Released - https://github.com/aws-amplify/amplify-ios/releases/tag/v1.22.0

authors: ameter charlie-luple royjit


text:
Fixed issue setting corporate proxy

Project failed to set proxy when http_proxy was available in env.
This PR fixes that.

Please I need this :(

repo: aws-azure-login/aws-azure-login


text:
Validation:  Resource property cannot set readOnly properties

Currently, if set a value for a readOnlyProperties, this will be silently ignored.  We should have validation to enforce this validation.

catching some of these because readOnlyProperties should never overlap with writeOnlyProperties or createOnlyProperties or required:
https://github.com/aws-cloudformation/cloudformation-cli/pull/663
https://github.com/aws-cloudformation/cloudformation-cli/pull/668

authors: PatMyron wbingli


text:
Graceful cancellation for network dependent commands

Issue N/A
ack-generate uses go-git to clone and fetch repository tags, these
actions are time consuming (more than a minute for cloning) and do
not tolerate interruption (ctrl+C). Forcing a command to exit when it's
running a git.Clone or a git.FetchTags cause the process to write
incomplete data on disk,  therefore re-running ack-generate will
always return an error. The only solution for this problem is to delete
the local sdk cache and let ack-generate reclone (without interruption)
This patch adds a graceful cancellation system for ack-generate
subcommads. Which will allow users to safely ctrl+C and re-run their
commands without any errors.
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

I see that it sets up a cancel context and prescribes timeouts to each git operation, but how does this prevent it from creating a faulty local version of the repositories?

I see that it sets up a cancel context and prescribes timeouts to each git operation, but how does this prevent it from creating a faulty local version of the repositories?

Yes on one hand this PR adds default timeouts for each git operation, on the other hand we also propagate a cancellation mechanism to all the functions using the main context (context.Background())
Short answer: It is up the go-git to properly handle context cancellation (which they do). We just have to properly ask go-git functions to cancel what they are doing before killing the process.
The cancellation mechanism doesn't guaranty that go-git will not write incomplete data and/or missing git blobs - but only ensure that we are correctly using go-git functions and any function that takes a context.Context as first parameter.
long answer:
Calling a cancel (function return by context.WIthCancel) sends a signal using the channel that is accessible via ctx.Done(). Every function that needs to implement graceful cancellation needs to listen for a ctx.Done() signal using a select statement.
How does go-git handle graceful cancellation for network calls and disk i/o operations ?
For disk IO operations, go-git uses a specific io.Writer implementations that listens to cancel signals  - note line 69 that is waiting for a ctx.Done() signal - which is part of the context.Context interface functions. (and still this implementation is not perfect since there no way to cancel a single write op using go stdlib)
For network calls - the cancellation mechanism is implemented in the standard libraries net, net/http, net/http/httputil ... some examples of handle cancel signals: https://github.com/golang/go/blob/2ebe77a2fda1ee9ff6fd9a3e08933ad1ebaea039/src/net/dial.go#L527-L558https://github.com/golang/go/blob/2ebe77a2fda1ee9ff6fd9a3e08933ad1ebaea039/src/net/dial.go#L384-L395

Glad go-git has support for cancellation contexts. Looks good!
/lgtm

@jaypipes If signalCh receives a message in L52 yes. However it's safe to call a context.CancelFunc more than once: https://github.com/golang/go/blob/master/src/context/context.go#L404

authors: A-Hilaly RedbackThomson


text:
(com.aws.greengrass.util.orchestration.SystemdUtils): Greengrass Nucleus Fails to start in WLS

Describe the bug
When executing Greengrass configuration inside Windows Linux Subsystem (WSL) it fails to start with the following error: Unable to setup Nucleus as system service
To Reproduce

Install WSL (Ubuntu 20.04) on Windows Server 2019 link
Install Java environment on WSL
Configure AWS credentials (as described here )
Download and install GeenGrass Nucleus (as described here )

sudo -E java -Droot="/greengrass/v2" -Dlog.store=FILE \
  -jar ./GreengrassCore/lib/Greengrass.jar \
  --aws-region us-east-1 \
  --thing-name GreengrassQuickStartCore-[id] \
  --thing-group-name GreengrassQuickStartGroup \
  --tes-role-name GGProvisioningRole \
  --tes-role-alias-name GGProvisioningRoleAlias \
  --component-default-user ggc_user:ggc_group \
  --provision true \
  --setup-system-service true

Expected behavior
It should start the service successfully
Actual behavior
Fail to start with error: Unable to setup Nucleus as system service
Environment

OS: Windows Linux Subsystem (Ubuntu 20.04) on Windows Server 2019
JDK version: 11
Nucleus version: 2.0.5

Additional context
GreenGrass error logs:
2021-03-12T03:45:16.804Z [ERROR] (Copier) com.aws.greengrass.util.orchestration.SystemdUtils: systemd-setup. {stderr=System has not been booted with systemd as init system (PID 1). Can't operate., command=systemctl daemon-reload}
2021-03-12T03:45:16.805Z [ERROR] (Copier) com.aws.greengrass.util.orchestration.SystemdUtils: systemd-setup. {stderr=Failed to connect to bus: Host is down, command=systemctl daemon-reload}
2021-03-12T03:45:16.807Z [ERROR] (main) com.aws.greengrass.util.orchestration.SystemdUtils: systemd-setup. Failed to set up systemd service. {}
java.io.IOException: Command systemctl daemon-reload failed
        at com.aws.greengrass.util.orchestration.SystemdUtils.runCommand(SystemdUtils.java:87)
        at com.aws.greengrass.util.orchestration.SystemdUtils.setupSystemService(SystemdUtils.java:46)
        at com.aws.greengrass.easysetup.GreengrassSetup.performSetup(GreengrassSetup.java:299)
        at com.aws.greengrass.easysetup.GreengrassSetup.main(GreengrassSetup.java:242)

If you remove the --setup-system-service true then it will be able to provision and start. The problem is that your WSL isn't setup using systemd as the error from systemctl says System has not been booted with systemd as init system (PID 1). Can't operate.
This seems to be a known issue with WSL: https://github.com/MicrosoftDocs/WSL/issues/457.
Once you've provisioned you will then want to manually configure to run Greengrass as a system service by calling the script located at <gg root>/alts/current/distro/bin/loader

If you do find an acceptable solution, please report back here and we can see about adding it either in documentation or into the code itself.

authors: MikeDombo mirgj

repo: aws-greengrass/aws-greengrass-nucleus


text:
feat: Helm chart secrets-store-csi-driver-provider-aws

What does this PR do?

Replaces https://github.com/aws/eks-charts/tree/master/stable/csi-secrets-store-provider-aws in favor of https://github.com/aws/secrets-store-csi-driver-provider-aws/tree/main/charts/secrets-store-csi-driver-provider-aws.
adds a check to the addon's namespace creation to not create kube-system.

Motivation

Resolves #1306

More

[x] Yes, I have tested the PR using my local account setup (Provide any test evidence report under Additional Notes)
[ ] Yes, I have added a new example under examples to support my PR
[ ] Yes, I have created another PR for add-ons under add-ons repo (if applicable)
[ ] Yes, I have updated the docs for this feature
[ ] Yes, I ran pre-commit run -a with this PR

Note: Not all the PRs require a new example and/or doc page. In general:

Use an existing example when possible to demonstrate a new addons usage
A new docs page under docs/add-ons/* is required for new a new addon

For Moderators

[ ] E2E Test successfully complete before merge?

Additional Notes
This is now working in my own eks cluster with the default secrets-store-csi-driver addon (enable_secrets_store_csi_driver = true). I have not thoroughly tested - I only have this working currently w/ no helm_config overrides passed to the addon module. Probably should be tested with namespace = "kube-system" for example.

@balakrishnasoura-okta I tried locally with the updated main.tf on this branch and still get this error. Something in the helm charts must have changed? secretproviderclasses-admin-role is now created in chart secrets-store-csi-driver and the second chart csi-secrets-store-provider-aws is still trying to create it? Sorry im pretty new to most of this so not great w/ debugging it.
module.tf_eks.module.csi_secrets_store_provider_aws.module.helm_addon.helm_release.addon[0]: Creating...
╷
│ Error: rendered manifests contain a resource that already exists. Unable to continue with install: ClusterRole "secretproviderclasses-admin-role" in namespace "" exists and cannot be imported into the current release: invalid ownership metadata; annotation validation error: key "meta.helm.sh/release-name" must equal "csi-secrets-store-provider-aws": current value is "secrets-store-csi-driver"; annotation validation error: key "meta.helm.sh/release-namespace" must equal "kube-system": current value is "secrets-store-csi-driver"
│
│   with module.tf_eks.module.csi_secrets_store_provider_aws.module.helm_addon.helm_release.addon[0],
│   on .terraform/modules/tf_eks.csi_secrets_store_provider_aws.helm_addon/modules/kubernetes-addons/helm-addon/main.tf line 1, in resource "helm_release" "addon":
│    1: resource "helm_release" "addon" {

I will troubleshoot by EOD today and update it in the PR

I have this working in an eks cluster now w/ no overrides passed to the addon module that is installing. I replaced the now-deprecated eks-charts/csi-secrets-store-provider-aws with github.com/aws/secrets-store-csi-driver-provider-aws. Probably needs more testing to be sure of this change, and not sure what else you want to update related to swapping out the chart.

Details

It seems the Title needs to be adjusted to show a version can you @atheiman do this to speed up the process? We are waiting for this to be fixed. Many thanks for the quick action

pre-commit output showing the following

Hello guys, any updates so far? Waiting for this to be merged 🤞

Is there a work-around for this issue. We really need this fixed.

Im not sure if they plan on merging this change or what still needs to be done b/c i think in some cases this could be a breaking change. I actually ended up not using the secretsmanager secret provider. But this workaround worked for me for a few days.
Most of the flags to enable features simply trigger additional sub modules of this tf project to be run. If you have any issue w/ how any of the submodules are configured, just copy them out of this project and run them as your own submodule that depends on your blueprints module call. So for this issue you could copy the secrets mgr secret store provider tf submodule out of my branch and put it in your codebase. Each submodule takes a long list of standard arguments about the cluster, most/all are available as outputs from the eks cluster module.
Unfortunately i dont have this workaround in place anymore so I dont have an example I can copy to show you, but hopefully this gets you down a path to a reasonable workaround.

thank you for the PR - this will be resolved in the new addons module https://github.com/aws-ia/terraform-aws-eks-blueprints-addons

authors: atheiman balakrishnasoura-okta bryantbiggs dannykruitbosch egarbi mchlbataller

repo: aws-ia/terraform-aws-eks-blueprints


text:
deployment guide is to be improved

Describe the issue
pls. improved the deployment guide, so that it's clear what to do and there're copy-paste commands for deployment
as an example pls. refer to the following links:
https://aws-samples.github.io/aws-genai-llm-chatbot/guide/deploy.html
https://aws-samples.github.io/aws-genai-llm-chatbot/guide/deploy.html#aws-cloud9
https://aws-samples.github.io/aws-genai-llm-chatbot/guide/deploy.html#deployment
Links
what I have currently is:

the first page - https://github.com/aws-samples/aws-genai-conversational-rag-reference
which refers to the full documentation link - https://aws-samples.github.io/aws-genai-conversational-rag-reference/overview/
the full documentation page has the 'Getting Started' chapter - https://aws-samples.github.io/aws-genai-conversational-rag-reference/overview/quick-start/
'Getting Started' chapter is poorly structured and and it takes time to understand how to deploy the proposed solution

Documentation improvements now work-in-progress

Please let me know what you think of the restructured documentation - adding clipboard affordances is will a WIP


text:
removing region from urls

Issue #, if available:
removing region from urls, and removing the warning about default region where applicable. (leaving it for the fluentd.yml file for now)
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

LGTM

authors: brentley testgitrz

repo: aws-samples/eks-workshop


text:
Extract variables to replace by environment var.

Issue #, if available:
Description of changes:
for a better understanding, Extract variables to replace with environment var. like my Cloudformation Stack name
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Same in this file :

https://github.com/aws-samples/eks-workshop/edit/master/content/x-ray/role.md

Better Fix with this PR https://github.com/aws-samples/eks-workshop/pull/229


text:
EC2 instance not being tagged

I've installed this into my aws account
 ❮❮❮ cdk deploy --require-approval never --parameters tags='{"map-migrated":""mig48715"}'

✨  Synthesis time: 5.41s

resource-tagging-automation:  start: Building 793ef7749effcfd2e39d437d007300ae7fbdf9a491c9cf979f7acb332d62eb55:current_account-current_region
resource-tagging-automation:  success: Built 793ef7749effcfd2e39d437d007300ae7fbdf9a491c9cf979f7acb332d62eb55:current_account-current_region
resource-tagging-automation:  start: Building 0ababf4706f4220e80c0f897b758e8f783c16508cbc725fd9e3d8a6b91b828aa:current_account-current_region
resource-tagging-automation:  success: Built 0ababf4706f4220e80c0f897b758e8f783c16508cbc725fd9e3d8a6b91b828aa:current_account-current_region
resource-tagging-automation:  start: Publishing 793ef7749effcfd2e39d437d007300ae7fbdf9a491c9cf979f7acb332d62eb55:current_account-current_region
resource-tagging-automation:  success: Published 793ef7749effcfd2e39d437d007300ae7fbdf9a491c9cf979f7acb332d62eb55:current_account-current_region
resource-tagging-automation:  start: Publishing 0ababf4706f4220e80c0f897b758e8f783c16508cbc725fd9e3d8a6b91b828aa:current_account-current_region
resource-tagging-automation:  success: Published 0ababf4706f4220e80c0f897b758e8f783c16508cbc725fd9e3d8a6b91b828aa:current_account-current_region
resource-tagging-automation: deploying... [1/1]
resource-tagging-automation: creating CloudFormation changeset...

 ✅  resource-tagging-automation

✨  Deployment time: 131.82s

Stack ARN:
arn:aws:cloudformation:us-west-2:716091368358:stack/resource-tagging-automation/28238ff0-0517-11ee-896d-022a18a03d41

✨  Total time: 137.23s

(.venv) 10:41AM /Users/paymahn/code/goldsky/resource-tagging-automation main ◼
 ❯❯❯ aws lambda list-functions | grep FunctionName | grep resource-tagging-automation-function

            "FunctionName": "resource-tagging-automation-function",

I then went and manually created an EC2 instance using the console. After making the instance and viewing the tags I don't see the my desired tag, map-migrated

when I view the function in lambda, I don't see any invocations

similarly, when I view the event bridge rule I don't see any invocations

What might be going on here?

It seems that the cloud trail -> event bridge connection isn't working properly. I can see the RunInstances events in cloud trail but it looks like those aren't being picked up by Event Bridge which then means that the Lambda doesn't get invoked.

thanks and I will add the issue in the doc.

authors: crazyoyo paymog

repo: aws-samples/resource-tagging-automation


text:
Alexa workshop fixes

Issue #, if available:
Fixes of the Alexa workshop for inclusion in the upcoming Retail Immersion Day
Description of changes:

remove of Amazon Pay
cosmetic changes on the Alexa Skill

Description of testing performed to validate your changes (required if pull request includes CloudFormation or source code changes):
walk through the workshop
By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Integration test on this branch: success

repo: aws-samples/retail-demo-store


text:
Pandas ValueError: operands could not be broadcast together with shapes (3,) (6,)

@szamarin Thank you for this very useful toolkit! It has already helped me verify some SageMaker Serverless Inference deployments. In some cases it is useful to execute run_serverless_benchmarks() with fewer than the full 6 memory size options. For example, when I tried this: memory_sizes=[4096, 5120, 6144] it would throw an exception from Pandas:

The problem originated here due to the hard-coded "6" in Line #318 of analysis.py:

This one-line change at Line #318 in analysis.py fixed the issue for running with 1-5 memory sizes:

Thanks again for a great piece of work!

Thanks for pointing this out @dkingram! I've released a new version which fixes this. Glad you're finding the tool useful and really appreciate your feedback.

Fixed in 0.2.2

authors: dkingram szamarin