text:
Running demo as-is, fails with  TF lookupTransform error

Hi,
I have cloned the package, built it catkin build --cmake-args -DBUILD_DEMO=True  and sourced it. After I launch it as-is, it fails and this is the output I get:
 source devel/setup.bash
asp@asp-MS-7B84:~/stitch_ws$ roslaunch yak_ros demo.launch
... logging to /home/asp/.ros/log/75a731ba-ec9b-11ea-804a-d037451d0501/roslaunch-asp-MS-7B84-7694.log
Checking log directory for disk usage. This may take a while.
Press Ctrl-C to interrupt
Done checking log file disk usage. Usage is <1GB.

started roslaunch server http://asp-MS-7B84:38869/

SUMMARY
========

PARAMETERS
 * /image_sim_node/base_frame: world
 * /image_sim_node/framerate: 30
 * /image_sim_node/mesh: /home/asp/stitch_...
 * /image_sim_node/orbit_speed: 1.0
 * /rosdistro: melodic
 * /rosversion: 1.14.9
 * /tsdf_node/camera_matrix: [550.0, 0.0, 320....
 * /tsdf_node/cols: 640
 * /tsdf_node/rows: 480
 * /tsdf_node/tsdf_frame: tsdf_origin
 * /tsdf_node/volume_resolution: 0.001
 * /tsdf_node/volume_x: 640
 * /tsdf_node/volume_y: 640
 * /tsdf_node/volume_z: 192

NODES
  /
    image_sim_node (yak_ros/yak_ros_image_simulator)
    link1_broadcaster (tf/static_transform_publisher)
    tsdf_node (yak_ros/yak_ros_node)

auto-starting new master
process[master]: started with pid [7704]
ROS_MASTER_URI=http://localhost:11311

setting /run_id to 75a731ba-ec9b-11ea-804a-d037451d0501
process[rosout-1]: started with pid [7715]
started core service [/rosout]
process[image_sim_node-2]: started with pid [7722]
process[tsdf_node-3]: started with pid [7723]
process[link1_broadcaster-4]: started with pid [7724]
[ WARN] [1598996303.698697205]: TF lookupTransform error: Lookup would require extrapolation at time 1598996303.697581838, but only time 1598996303.735359645 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1598996303.731441946]: TF lookupTransform error: Lookup would require extrapolation at time 1598996303.697581838, but only time 1598996303.735359645 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1598996303.760431768]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1598996303.697581838 but the earliest data is at time 1598996303.735359645, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1598996303.797623405]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1598996303.730902085 but the earliest data is at time 1598996303.735359645, when looking up transform from frame [camera] to frame [tsdf_origin]
[ERROR] [1598996303.910701637]: Failed to fuse image.

Blank result:

this is my rviz configuration.

I have done all I can to make sure Cuda is working, so it is out of the way.
I could run the cuda examples with no problem.
this is an example of CUDA working:

I tried to integrate the package to our industrial robot, I have been getting similar errors. Therefore I went to square one to make sure I can first get a mesh out of the demo first.
Please advise, thank you very much. :D

I'll do more investigation, I was able to make the demo work in a jetson tx2.

Can you provide some more info about the hardware and environment where it isn't working? The most important data is:

Operating system
CUDA version
GPU type

It's definitely good that you were able to get CUDA running independently, since it's probably the main source of weird issues related to getting KinectFusion-type applications up and running. The fact that your Rviz displays the depth image correctly shows that the simulated depth image pipeline is working too.

I'll investigate more, I was able to make the demo work in a jetson tx2.

This is a little encouraging, since I think it narrows down the problem to something specific to your first computer.

Operating system: Ubuntu: 18.04.5 LTS (Bionic Beaver)
NVIDIA Corporation GK107 [GeForce GT 740]
asp@asp-MS-7B84:~$ nvcc --version
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85

I realized when I run
/usr/local/cuda/bin/nvcc --version
it gives:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

which is different from the previous nvcc --version command. I will look into this more
lastly:
'NVIDIA Driver Version: 450.51.06'

If you have multiple versions of CUDA installed, the problem might be that your workspace is building against one version but trying to run using the other version and failing. Do you have anything in your .bashrc or elsewhere that adds CUDA paths to LD_LIBRARY_PATH or other environment variables? If so, which version of CUDA do they point to?
The GK107 GPU is listed as Compute Capability 3.0 (Kepler) which means it should be compatible with either CUDA 9.1 or 10.2 as well as Nvidia Driver 450.51, so there shouldn't be anything fundamentally preventing this software from running on your computer. This compatibility info is listed in CUDA's documentation.

It seemed that after installing cuda, we installed cuda_tools from apt-ge at a different version?
I uninstalled Cuda completely then reinstalled cuda 10.2, with driver 390
I didnt install cuda-tools ( nvc --version) gives
Command 'nvcc' not found, but can be installed with:

sudo apt install nvidia-cuda-toolkit

However,  /usr/local/cuda/bin/nvcc --version gives:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

File ~/.bashrc includes:
export PATH=:...:/usr/local/cuda-10.2
#cuda environment variables
export LD_LIBRARY_PATH=:usr/local/cuda-10.2/lib64:

I have built and tested the demo with and I received same errors.
[ WARN] [1599576968.271444562]: TF lookupTransform error: Lookup would require extrapolation at time 1599576968.270284721, but only time 1599576968.273002273 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1599576968.300546820]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1599576968.270284721 but the earliest data is at time 1599576968.273002273, when looking up transform from frame [camera] to frame [tsdf_origin]
[ERROR] [1599576968.442203459]: Failed to fuse image.

I upped the update rate on demo.lunch file
<param name="framerate" value="90"/>

The picture was still not fused on the output windows
Lastly, we have upgraded the Nvidia GPU hardware to NVIDIA Corporation GM204 [GeForce GTX 970] which has Maxwell architecture and the demonstration worked right away (with the previous CUDA and driver configurations) .
The demonstration still gave warning and an error message, which I am not sure of the severity of.
[ WARN] [1599576968.271444562]: TF lookupTransform error: Lookup would require extrapolation at time 1599576968.270284721, but only time 1599576968.273002273 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1599576968.300546820]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1599576968.270284721 but the earliest data is at time 1599576968.273002273, when looking up transform from frame [camera] to frame [tsdf_origin]
[ERROR] [1599576968.442203459]: Failed to fuse image.

However, there we can see the fused-image on the output window which is definitely a good sign.
.

I uninstalled Cuda completely then reinstalled cuda 10.2, with driver 390

How did you pick which version of the Nvidia driver to install? I don't think that driver 390 is compatible with CUDA 10.2, since the compatibility chart says you need at least driver 440.33. There's some info on CUDA setup in the Yak readme. If you run ubuntu-drivers devices it will list the possible Nvidia driver versions that are compatible with your GPU.

I didnt install cuda-tools ( nvc --version) gives
Command 'nvcc' not found, but can be installed with:

sudo apt install nvidia-cuda-toolkit

However, /usr/local/cuda/bin/nvcc --version gives:
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2019 NVIDIA Corporation
Built on Wed_Oct_23_19:24:38_PDT_2019
Cuda compilation tools, release 10.2, V10.2.89

File ~/.bashrc includes:
export PATH=:...:/usr/local/cuda-10.2
#cuda environment variables
export LD_LIBRARY_PATH=:usr/local/cuda-10.2/lib64:

I think this is just an environment setup issue. Make sure you follow the post-installation environment setup instructions in section 6.1.1 of the CUDA installation guide, and make sure that the paths you're adding to the environment variable are correct.

I have built and tested the demo with and I received same errors.
[ WARN] [1599576968.271444562]: TF lookupTransform error: Lookup would require extrapolation at time 1599576968.270284721, but only time 1599576968.273002273 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1599576968.300546820]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1599576968.270284721 but the earliest data is at time 1599576968.273002273, when looking up transform from frame [camera] to frame [tsdf_origin]
[ERROR] [1599576968.442203459]: Failed to fuse image.

I upped the update rate on demo.lunch file
<param name="framerate" value="90"/>

The picture was still not fused on the output windows
Lastly, we have upgraded the Nvidia GPU hardware to NVIDIA Corporation GM204 [GeForce GTX 970] which has Maxwell architecture and the demonstration worked right away (with the previous CUDA and driver configurations) .
The demonstration still gave warning and an error message, which I am not sure of the severity of.
[ WARN] [1599576968.271444562]: TF lookupTransform error: Lookup would require extrapolation at time 1599576968.270284721, but only time 1599576968.273002273 is in the buffer, when looking up transform from frame [camera] to frame [tsdf_origin]
[ WARN] [1599576968.300546820]: TF lookupTransform error: Lookup would require extrapolation into the past.  Requested time 1599576968.270284721 but the earliest data is at time 1599576968.273002273, when looking up transform from frame [camera] to frame [tsdf_origin]
[ERROR] [1599576968.442203459]: Failed to fuse image.

However, there we can see the fused-image on the output window which is definitely a good sign.

The warnings aren't a big deal, it just means that the TSDF node received an image before TF data was available for the camera frame. The error can indicate a problem with the CUDA portion of the fusion algorithm, but if you're getting correct output from the demo node then it's also not something to worry about in this case.
Since you were able to get it working by swapping out your GPU, I think your original issues were caused by having an incompatible combination of GPU hardware, Nvidia driver, and CUDA version (going back to my first comment about checking for the recommended Nvidia driver for your GPU). The CUDA-dependent packages will still compile correctly as long as compatible CUDA libraries are available, but runtime errors like the ones you've encountered will still appear if the GPU hardware or Nvidia driver isn't compatible with the CUDA libraries used to build the packages.

Thank you for your help and explanations @schornakj :
Indeed, I think some of the errors were not following the post-installation instructions exactly, and incompatible combination of GPU hardware, Nvidia driver, and CUDA version.
Now the ~/.bashrc incldues
export PATH=:...:/usr/local/cuda-10.2/bin${PATH:+:${PATH}}
#cuda environment variables
export LD_LIBRARY_PATH=:usr/local/cuda-10.2/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

afterwards, I notice the nvcc --version works fine in a new tab.
I rechecked the driver now and it is at 450.51.06 for clarity.
It was a bit confusing to read (more than or less than) the driver versions compatibility with CUDA versions.
Warmest Regards

I'm really glad to hear that it's running now! Feel free to post another issue if you have other questions. We would definitely be interested in hearing more about your project in general as well.

It was a bit confusing to read (more than or less than) the driver versions compatibility with CUDA versions in the cuda documentation.

I think this is probably the hardest part of setting up this type of system. The first time I had to install CUDA on a computer it took me an entire week to get it working!

authors: Beshario schornakj


text:
Fix shutdown of soccer_ipm

When a the software is interrupted, a KeyboardInterrupt is raised in ex.spin(). This should be excepted instead of being raised to the user and the node should be destroyed. rclpy.shutdown() should not be called because the keyboard interrupt already does the shutdown. (If we don't remove it, we get the error failed to shutdown: rcl_shutdown already called on the given context).

Thanks @timonegk !

source: gharchive/pull-request

authors: ijnek timonegk

license_type: permissive

license_source: github-api


text:
ROS2: Publish an array with rqt

Hi everyone, I have been using rqt to publish messages to test my applications, however I still haven't found a way to publish a message that includes an array.
In my custom message I have a variable size array declared as uint8[]. I notice in rqt that the expected type is "sequence".
I have tried different expressions, but couldn't figure out the correct one. Could someone provide an example please?

If we assume get_slot_types functions correctly, then my fix here allows us to publish arrays.  However, as mentioned in this related ticket, a much larger PR should probably be merged first, before addressing this issue.
If anyone has any guidance on the guidance to help get these changes through, I'll be happy to help where fit.

As porposed in #34 i tried to compile the rolling release and still got the error
Traceback (most recent call last):
  File "/colcon_ws/install/rqt_publisher/lib/python3.10/site-packages/rqt_publisher/publisher.py", line 158, in change_publisher
    new_text = handler(self._publishers[publisher_id], topic_name, new_value)
  File "/colcon_ws/install/rqt_publisher/lib/python3.10/site-packages/rqt_publisher/publisher.py", line 257, in _change_publisher_expression
    success, _ = self._evaluate_expression(computed_expression, slot_type)
  File "/colcon_ws/install/rqt_publisher/lib/python3.10/site-packages/rqt_publisher/publisher.py", line 376, in _evaluate_expression
    if successful_eval and isinstance(value, slot_type):
TypeError: isinstance() arg 2 must be a type, a tuple of types, or a union

after i typed array('I',[1,2,3])
or got this error if i used array.array('I',[1,2,3])
[ERROR] [1673339078.788973249] [get_message_class]: Malformed message_type: sequence<uint32>
Python eval failed for expression "array.array('I',[1,2,3])" with an exception "type object 'array.array' has no attribute 'array'"
Publisher._evaluate_expression(): failed to evaluate expression: "array.array('I',[1,2,3])" as Python type "None"
Traceback (most recent call last):
  File "/colcon_ws/install/rqt_publisher/lib/python3.10/site-packages/rqt_publisher/publisher.py", line 158, in change_publisher
    new_text = handler(self._publishers[publisher_id], topic_name, new_value)
  File "/colcon_ws/install/rqt_publisher/lib/python3.10/site-packages/rqt_publisher/publisher.py", line 274, in _change_publisher_expression
    user_expression, slot_type.__name__)
AttributeError: 'NoneType' object has no attribute '__name__'. Did you mean: '__ne__'?

If i understood the other Issue #34 correctly this should have worked? I am using a UInt32 MultiArray

There is still no way, or at least no documented way on doing this. Can anybody help out?

Right now there is no way to do this since rqt_publisher does not support the array.array type and the common libraries don't support sequence<X> builtin types. I just submitted #39 to humble to fix both of these.
If you want to use it, strings are represented as normal lists, e.g.: ["joint_1", "joint_2"]
Numerical arrays are represented as array types, e.g.: array('d', [5.5, 4.5]) or array('i', [3, 2]).
See a complete list of numerical typecodes at https://docs.python.org/3/library/array.html

source: gharchive/issue

authors: George-Gi RyanDelGizzi baldhat egordon wienans


text:
Rename DynreconfClientWidget => ParamClientWidget

@gonzodepedro, it would be great if you could target other changes like this one from the ROS 2 PR that you feel are appropriate to bring back to the ROS 1 branch.
When this one gets merged I'll do the big  merge back to see how much smaller the ROS 2 PR got after  all of these changes.

This is good feedback.
I thought Param was appropriate because within Dynamic Reconfigure, the individual values are still referred to as parameters. But you're right that they aren't formal parameters in ROS 1 like they are in ROS 2.
Is there another more neutral term we could use? There is still a lot of crossover in code and purpose between the DynreconfClientWidget in ROS 1 and the ParamClientWidget you made in ROS 2.

I really cannot think of a better name. I went thought the code and in the ros1 version we still refer to the "dynamic reconfigurable" items as "params". So LGTM

authors: cottsay gonzodepedro


text:
Possible issue with interactive marker control with local (moving) coordinate frame

Hi,
When using interactive markers with MOVE_3D and MOVE_ROTATE_3D option, I found that when the marker is clicked, the marker jumps. This happens when the marker is attached to a local coordinate frame (specified header.frame_id in InteractiveMarker) which is different from Fixed Frame specified in rviz. The amount of jump is related to the difference between the fixed frame and the local coordinates.
My guess is that this is possibly handled in moveViewPlane() in interactive_maker_control.cpp in default_plugin in rviz. With the MOVE_PLANE option, this jump does not occur. Also, when the ring and arrow are clicked with ROTATE_AXIS and MOVE_AXIS option, the jump does not happen.
I would most appreciate it if you could investigate this issue and provide a possible solution to it. I'm using ROS Hydro on Ubuntu 12.04.
The same issue has been posted in the ROS forum
http://answers.ros.org/question/138616/rviz-tf-and-interactivemarker-pose-with-respect-to-which-frame/
Follow up:
I was trying to find where this jump is happening, and most likely this function in moveViewPlane() is causing the jump.
// move parent so grab position relative to parent coincides with new mouse position.
parent_->setPose( mouse_position_on_plane - parent_to_grab_position_,
parent_->getOrientation(),
name_ );
and also it would be great if you could check these in beginMouseMovement()
grab_point_in_reference_frame_ = reference_node_->convertWorldToLocalPosition(grab_point_in_world_frame);
parent_to_grab_position_ = grab_point_in_world_frame - parent_->getPosition();
Best regards,
-Jun

Hi,
Also, this could be related to beginMouseMovement() in handleMouseEvent(). So, it would be great if you could look into the coordinate transform between the current frame and the fixed frame.
Thank you,
-Jun

Hi,
I further looked into this issue, and seemingly setting
parent_to_grab_position_  = Ogre::Vector3(0,0,0);
solved this problem and looks like it is working fine. So, either by setting parent_to_grab_position_ to zero, or changing the following two functions as follows might be a possible solution (removing  parent_to_grab_position_ from these).
// find plane on which mouse is moving
Ogre::Plane plane( event.viewport->getCamera()->getRealDirection(),
parent_position_at_mouse_down_);
parent_->setPose( mouse_position_on_plane,
                  parent_->getOrientation(),
                  name_ );

Your confirmation would be much appreciated!
Best regards,
-Jun

Hi,
Further to my previous post, modification of moveViewPlane() in the following way achieved the behavior that I had expected:
Ogre::Vector3 camera_direction_ = event.viewport->getCamera()->getRealDirection();
Ogre::Plane plane( reference_node_->getOrientation().Inverse()*camera_direction_,
                   parent_position_at_mouse_down_);

and
parent_->setPose( mouse_position_on_plane ,
                  parent_->getOrientation(),
                  name_ );

Best regards,
-Jun

Crossreference: https://github.com/ros-visualization/rviz/pull/937#issuecomment-200076436

@jjnak Could you please check, that this issue is resolved by #961. You describe exactly the same behaviour I observed too.

authors: jjnak rhaschke wjwwood

repo: ros-visualization/rviz

license_source: bigquery


text:
ROS2 on Windows, #warning is not recognized by VS

In this deprecated header: https://github.com/ros/pluginlib/blob/9b319f1440a7153d6edf68067093449ff7b14201/pluginlib/include/pluginlib/class_list_macros.h#L36
Possible solutions:

Use #pragma message("...")
Surround #warning checking for compiler
Remove the deprecated header and break things

More discussion about solutions
https://stackoverflow.com/questions/2685598/warning-in-visual-studio

I think we removed this a long time ago, so going ahead and closing this one.

authors: brawner clalancette

repo: ros/pluginlib


text:
add pyqrcode from pip

as available here: https://pypi.org/project/PyQRCode/

Is this the same package as https://packages.ubuntu.com/bionic/python-pyqrcode ?
If so please use the apt version rather than pip for the disctributions where it's available

I also added the packages for
arch: https://www.archlinux.org/packages/?sort=&q=qrcode&maintainer=&flagged=
debian: https://packages.debian.org/search?suite=default&section=all&arch=any&lang=de&searchon=names&keywords=python-qrcode
gentoo: https://packages.gentoo.org/packages/dev-python/pyqrcode

@ipa-fxm thanks for iterating.
Please see review of https://github.com/ros/rosdistro/pull/17950 for details about:

how to add keys in a future proof way
not adding entries for EOL distros
providing pip keys for debian distributions that don't have the package

I hope everything looks good now

authors: ipa-fxm mikaelarguedas

repo: ros/rosdistro


text:
rosidl_typesupport: 1.2.0-1 in 'rolling/distribution.yaml' [bloom]

Increasing version of package(s) in repository rosidl_typesupport to 1.2.0-1:

upstream repository: https://github.com/ros2/rosidl_typesupport.git
release repository: https://github.com/ros2-gbp/rosidl_typesupport-release.git
distro file: rolling/distribution.yaml
bloom version: 0.10.3
previous version for package: 1.1.2-1

rosidl_typesupport_c
* Remove dependencies from Connext type support (#106 <https://github.com/ros2/rosidl_typesupport/issues/106>)
* Contributors: Andrea Sorbini

rosidl_typesupport_cpp
* Remove dependencies from Connext type support (#106 <https://github.com/ros2/rosidl_typesupport/issues/106>)
* Contributors: Andrea Sorbini

Please hold for coordinated merge...


text:
qt_gui_core: 2.1.0-1 in 'rolling/distribution.yaml' [bloom]

Increasing version of package(s) in repository qt_gui_core to 2.1.0-1:

upstream repository: https://github.com/ros-visualization/qt_gui_core.git
release repository: https://github.com/ros2-gbp/qt_gui_core-release.git
distro file: rolling/distribution.yaml
bloom version: 0.10.7
previous version for package: 2.0.1-1

qt_dotgraph

No changes

qt_gui

No changes

qt_gui_app

No changes

qt_gui_core

No changes

qt_gui_cpp
* FindPython3 explicitly instead of FindPythonInterp implicitly (#254 <https://github.com/ros-visualization/qt_gui_core/issues/254>)
* Contributors: Shane Loretz

qt_gui_py_common

No changes

This pull request has been mentioned on ROS Discourse. There might be relevant details there:
https://discourse.ros.org/t/preparing-for-rolling-sync-2021-10-29/22891/3

authors: clalancette ros-discourse


text:
rtabmap_ros: 0.20.18-1 in 'galactic/distribution.yaml' [bloom]

Increasing version of package(s) in repository rtabmap_ros to 0.20.18-1:

upstream repository: https://github.com/introlab/rtabmap_ros.git
release repository: https://github.com/introlab/rtabmap_ros-release.git
distro file: galactic/distribution.yaml
bloom version: 0.10.7
previous version for package: 0.20.16-2

Note: merge https://github.com/ros/rosdistro/pull/31970 before this one.


text:
Migration guide for parameters

I'm not aware of any existing resource about the changes for parameters going from ROS 1 to ROS 2. In particular, I think it would be useful if there was at least a brief about how to update existing parameter YAML files to be ROS 2 compatible. Perhaps as part of the Migration Guide. I don't know the details of the necessary changes, but I believe there are some.

I can help closing this issue. I read the design document for the change between the past Parameter Server vs the new parameters assigned to nodes functionality and there is a brief section explaining how to implement a similar feature in ROS 2. https://design.ros2.org/articles/ros_parameters.html
What could be the best example scenario for the Migration Guide? Maybe it will be needed an additional document with the command line differences for "ros2 param" vs "ros param", WDYT? @jacobperron
If there was needed an additional document to outline the differences, where do you think it should go? @maryaB-osr

@Blast545 Great!
Here are some things that I think are worth adding to the migration guide:

Summarizing the difference between the ROS 1 parameter server model and how parameters are tied to nodes in ROS 2

we can add a reference to the design doc for more details.

Differences between a ROS 1 parameter YAML file and the ROS 2 parameter YAML file.

E.g. there are some small changes users should add like the node name(s) and ros__parameters key.

Briefly mention some new features, like passing parameters on the command-line (ie. individual parameters or a parameters file)

I don't think we should explain how to re-create the parameter server from ROS 1.
Also, I don't think we need to describe differences between the command line tools (ros2 param vs ros param); there's a whole tutorial for that, which we can simply reference in the migration guide.
In the Migration Guide, there is a reference to a separate migration page for launch files. I think we could do something similar for parameters; create a separate article for migrating ROS 1 parameters to ROS 2.

Done in https://github.com/ros2/ros2_documentation/pull/887


text:
Add documentation of the scope of binary packaging jobs

The install from Binary instructions such as https://index.ros.org/doc/ros2/Installation/Foxy/#binary-packages does not mention that the binary packages for windows does not include all packages in Foxy.
When this was first written all packages were in the repos files, but now there are other packages that are not included in the binary packages, so we should document what the scope of packages is that are packaged.
As a related discussion right now we have only one set of packaging jobs. However we have several clearly defined entry points that we're managing https://github.com/ros2/variants in REP 2001 https://ros.org/reps/rep-2001.html And we package docker images of those variants. it might make sense for us to setup our packaging jobs to match those instead of documenting a 3rd non-standard variant.
We should also document how to add more of the released packages to your overlay workspace using tools like rosinstall_generator.
This is a follow up to: https://answers.ros.org/question/354341/using-xacro-package-in-ros-2/ demonstrating the confusion.

We discussed this in the ROS team meeting.
Since Foxy and earlier are already out we will add documentation for the existing ros2.repos scope.
Long term we would like to move away from the ros2.repos file and to that end in the upcoming rolling release we will plan to migrate to use a variant based replacement.
We will either:

Create a new variant to replicate the current ros2.repos scope
Confirm that an existing variant is appropriate, or adjust it to match.

See https://github.com/ros2/ros2_documentation/pull/802, which adds a note to the top of installation instructions involving fat archives setting expectations of what packages will be installed.

authors: jacobperron tfoote


text:
Warning while installing ROS2 on Ubuntu 18.04 with ROS1 melodic

When I'm following the installation guide for Install dependencies using rosdep, the following warning occurs.
~/ros2_dashing$ rosdep install --from-paths src --ignore-src --rosdistro dashing -y --skip-keys "console_bridge fastcdr fastrtps libopensplice67 libopensplice69 rti-connext-dds-5.3.1 urdfdom_headers"

WARNING: given --rosdistro dashing but ROS_DISTRO is "melodic". Ignoring environment.

Is there any specific instruction for installing both the ROS1 and ROS2 on a system? Would this affect my ROS1 melodic setup?
How to specify the dependencies so that my existing ROS1 melodic is unaffected?

WARNING: given --rosdistro dashing but ROS_DISTRO is "melodic". Ignoring environment.

That warning is saying that you have sourced melodic (usually /opt/ros/melodic/setup.bash), but then you are trying to use dashing when using rosdep install.  In general, we recommend that you only source the ROS distribution that you want to use in the particular console window that you are using.  Anytime you source/use multiple different ROS distributions in the same console, you'll get this warning.

Is there any specific instruction for installing both the ROS1 and ROS2 on a system? Would this affect my ROS1 melodic setup?

No, it doesn't affect the setup of Melodic at all.
In general, please ask questions like this on https://answers.ros.org.  That is a single place where this conversation can be searched for and found for people seeing the same problem in the future.
I'm going to go ahead and close this issue.

authors: Jaroan clalancette


text:
Disable windows re-org

Hi!
Thanks for this awesome plugin and theme! I recently moved to Rose Pine for my default theme. but I would like to change the how windows are shown in the current plugin.
With my default tmux config this what I get:

List of window, the selected one have * before it.
But with rose-pine tmux, the list is dynamic and changes according to current focused window.

I find this really confusing and I would like this list to be static except the change of selected one.
Is it possible to configure this option and disable it?

Can you explain in what way are they getting "reorganised"? The only thing the plugin should do is highlight differently the current one.

@mrs4ndman I will elaborate:
In my plain tmux config, when I select a window it just showed as 2:another-win* (as you can see in the first screenshot) and then to go to "yosi" named window, it's still the first-window.
With rose-pine, we are in the same state, two windows are opened, and I am "another-window", but:

another-window moved to the left
and "yosi" moved to right

As for the option, it would mean rebuilding the window-displaying code and adding the toggle would be the least troublesome part

If this is troublesome, I can do this change in my own fork if you can instruct me on how to achieve this.

When using this config for the plugin:
set -g @plugin 'rose-pine/tmux'
set -g @rose_pine_variant 'main'

Does it keep happening? I am testing trying to recreate the undesired window movement (I don't know what is causing it at the moment)

Thanks to this issue, I have re-thought the plugin's options, so the next commit(s) will implement the options to change the default behaviour. Without setting anything, the windows will behave the same as if there was no plugin.

Does this look cool as default behavior?

This afternoon I will probably update the README, and it has a high probability of implementing breaking changes, so I might create an issue for those to track them

@yosiat Having these options inside the config leads to the default tmux behaviour of active & non-active windows:
set -g @rose_pine_variant 'main'
set -g @rose_pine_show_current_program 'off'

I haven't had problems with renaming windows, although it might be a bug / oversight on my side.

Closing because the original issue has subsided :smiley:

repo: rose-pine/tmux

license: MIT


text:
Travis CI configuration

Replacing the ROS buildfarm CI tests with Travis CI

Well, I'm not sure whether we want to do this now or not. It looks like the ROS buildfarm stuff is working again (it's stopped trying to run the unsupported lunar builds). The method I'm using here runs the same scripts as the buildfarm, but in a docker container. So we can't test lunar anyway, and we can't test melodic until it's been released. This version takes about 20 minutes to run the test, the ROS buildfarm version takes about 11 minutes.
Should we just keep the buildfarm for now? The only annoying thing is that next time they drop support for a ROS distro, our CI system will probably be broken for a while again.

I think this is better than Jenkins.  I haven't enjoyed being subject to issues with the Jenkins server.
Having our own Travis does seem to take a 4-minute hit in terms of time to complete the CI.  I don't think that's a huge deal, because Jenkins takes 11 minutes (or in some of these more recent builds, a month).
If I'm going to wait 11 minutes, four more isn't a huge deal.

authors: dpkoch superjax

repo: rosflight/rosflight


text:
feat: allow to specify a different repository to deploy to

Resolves #41
Description
Thank you for creating this fantastic action!
I've added a feature to deploy previews to GitHub Pages of another repository. The following inputs have been added:

deploy-repository ... Specify the repository name to deploy. This is an input that can be passed to JamesIves/github-pages-deploy-action. The default is github.repository.
token ... The default GITHUB_TOKEN does not have enough permissions since it will be pushing to another repository. Therefore, the ability to pass a token is added.

These inputs are passed to JamesIves/github-pages-deploy-action.
Also, the pagesurl can be changed to the repository to deploy to.
Motivation
On the site I manage, I deploy to GitHub Pages using the beta version of GitHub Actions.
As a result, I'm facing the same problem as in this issue: https://github.com/rossjrw/pr-preview-action/issues/21, and I would like to deploy the preview environment to another repository as a workaround.
Verification
I have prepared the following two repositories and verified the operation:

source repo ... https://github.com/MH4GF/test-pr-preview-source

workflows ... https://github.com/MH4GF/test-pr-preview-source/actions/workflows/deploy-preview.yml

target repo ... https://github.com/MH4GF/test-pr-review-target
preview(target repo) ... https://mh4gf.github.io/test-pr-review-target/pr-preview/pr-3/
preview(source repo, check regression) ... https://mh4gf.github.io/test-pr-preview-source/pr-preview/pr-3/

Looks good to me! I've made some documentation changes. Thanks for the contribution @MH4GF!

Released in v1.4.0 (https://github.com/rossjrw/pr-preview-action/releases/tag/v1.4.0)

repo: rossjrw/pr-preview-action


text:
IE11 Errors vs. Only Expanded

Hi @roszpun,
I was getting these strange errors on IE11 while running v. 1.0.11:

After upgrading to v. 1.0.13, the errors are gone, but the styles from wrapper.vue don't get loaded.
I solved the issue by copying these wrapper styles into my component:
<style>
    .v-collapse-content{
        max-height: 0;
        transition: max-height 0.3s ease-out;
        overflow: hidden;
        padding: 0;
    }

    .v-collapse-content-end{
        transition: max-height 0.3s ease-in;
        max-height: 500px;
    }
</style>

...but that's obviously not ideal.
Maybe there's a broken path somewhere?

Using the latest version, I also had to get the styles into my SASS stylesheet. They were not being used automatically.

repo: roszpun/vue-collapse


text:
Too sensitive to hide / show

It's too sensitive that any scrolling will hide / show the bottom bar, is it possible to add some buffer, e.g. scroll at least 4 px before hiding / showing?

@xizzhu Yes, thats possible, but the MD guidelines do not specify such buffer, contrary it says that it suppose to hide as soon as user scrolls down and shows up as soon as user scrolls up.

authors: NikolaDespotoski xizzhu

repo: roughike/BottomBar


text:
Development

Hello! I've added BottomBar to my Android app Speed Boost available here: https://play.google.com/store/apps/details?id=com.TMillerApps.SpeedBoostLite&hl=en

Coverage increased (+0.2%) to 56.098% when pulling ef0dffd8efed848d31f473ed7ccb6b0683797624 on development into 31181165063355880215512e77f55a69884dc371 on master.


text:
create new image with background fails

I have tried to create a simple image with a black background but that didn't work, the image was not created but no error was returned
%Mogrify.Image{}
|> Mogrify.custom("size", "280x280")
|> Mogrify.custom("background", "#000000")
|> Mogrify.create(path: "/tmp/new.jpg")

After some investigation it looks like the background is not accepted and the argument should be canvas instead:
https://stackoverflow.com/questions/39504522/create-blank-image-in-imagemagick
Using canvas instead didn't work either
%Mogrify.Image{}
|> Mogrify.custom("size", "280x280")
|> Mogrify.custom("canvas", "#000000")
|> Mogrify.create(path: "/tmp/new.jpg")

Is this a bug?
If so do you want a PR?

Which ImageMagick version do you have installed? This library generally targets v6. v7 may break some compatibility.

Oh, that's why then, I am using v7.
Thank you

authors: angelo-moreira talklittle

repo: route/mogrify

license: mit


text:
Reconnect to the cloud when credentials expire

Heavily inspired by https://github.com/dahlb/ha_hatch/blob/a838a755a84df47de148db767b839693e04726f2/custom_components/ha_hatch/init.py#L129

Looks good! At some point, we should add more type annotations to cloud and api (and then maybe start using mypy too).

I'm trying to use dataclasses where I can not to rely on dicts

repo: rowysock/home-assistant-HomeWhiz


text:
🛑 MK Renovation & Remodel is down

In dc1379a, MK Renovation & Remodel (https://mkrenovationremodel.com/) was down:

HTTP code: 0
Response time: 0 ms

Resolved: MK Renovation & Remodel is back up in 3a10a61 after 15 minutes.

repo: rpharaniya/websites-uptime-monitor


text:
Using functions in sequence

Hi,
I like this module a lot, I am using it a lot. But I have a question which popped up when I was writing the sequence a little bit more structured. Maybe (I hope so) it's just a question of the syntax.
How to write it, when I would like to "call a function" in the sequence, e.g. a modification of the example from the README:
def firstSeq():
    VDD.setRiseFall(10e-6, 10e-6),
    VDD.applyV(5.0),
    va.WaitUs(10),
    CLK.on(4e6),
    va.WaitUs(1),

#Sequence
mod.seq(True)(
    firstSeq(),
    RST.write(True),
    va.WaitSignal(EVNT_READY),
    OUT.setRiseFall(100e-9, 100e-9),
    OUT.applyI(100e-3), #Positive current enters the model
    va.WaitUs(10),

```    CONFIG_VOUT.write(5),
    va.WaitUs(20),
    va.Finish()
)

Many thanks for the module and all your support!
KR,
Christof

Hello Christof,
I'm glad to here that the module is useful to you.  Sequences accept two types of classes: veriloga.Cmd and veriloga.CmdList. As the name suggests, veriloga.CmdList holds a list of veriloga.Cmd objects. Therefore, one way to implement what you need is:
`
def firstSeq():
return va.CmdList(
VDD.setRiseFall(10e-6, 10e-6),
VDD.applyV(5.0),
va.WaitUs(10),
CLK.on(4e6),
va.WaitUs(1)
)
#Sequence
mod.seq(True)(
firstSeq(),
RST.write(True),
va.WaitSignal(EVNT_READY),
OUT.setRiseFall(100e-9, 100e-9),
OUT.applyI(100e-3), #Positive current enters the model
va.WaitUs(10),
CONFIG_VOUT.write(5),
va.WaitUs(20),
va.Finish()
)
`
KR,
Rodrigo

Thanks a lot for the fast response, this way it works!


text:
Add some event modal form input controllers.

For example, add a checkbox for stating that the event is all day. This would hide the starts at and ends at inputs.
If the event's frequency is set to ONCE: hide the recurrence day/week/month inputs, hide the separation input and set its value to 1.
And other stuff like that.

Once

hide separation
hide day
hide week
hide month
set separation value to 1

Daily

hide day
hide week
hide month

Weekly

hide week

hide month

Monthly

repo: rrickgauer/DotNetTasks


text:
Use canonical URLs to manage search engine indexes

We do not want Google to serve up a search result with ?ref=elixirweekly in the URL, firstly because that's false (it's not an EW referral if a SERP provides it) and secondly because it splits ElixirStatus's Google juice between two identical pages.

repo: rrrene/elixirstatus-web


text:
Property 'TrimLeft' does not exist on type 'string'.ts(2339)

at snapshot.ts 117line, can not build

Not sure what happens here, could you confirm your typescript version and tsconfig?
The build process works fine in my local dev env and CI.

Please reopen this issue if there is any further information.

repo: rrweb-io/rrweb-snapshot


text:
Animated WebP Image not showing correctly using UIImageView or FLAnimatedImageView

New Issue Checklist

[x] I have read and understood the CONTRIBUTING guide
[x] I have read the Documentation
[x] I have searched for a similar issue in the project and found none

Issue Info

Info
Value

Platform Name
ios

Platform Version
7.0-10.0

SDWebImage Version
4.0

Integration Method
cocoapods

Xcode Version
Xcode 8.3.2

Repro rate
all the time (100%)

Repro with our demo prj
YES, the demo Image-5 renders wrong.

Demo project link
https://github.com/dreampiggy/AnimatedWebPDemo

Issue Description and Steps
Description:
SDWebImage 4.0 add support to animated WebP. But when downloading and showing some of animated WebP using SDWebImage API(eg. sd_setImageWithURL:) to UIImageView or FLAnimatedImageView instance, the imageview can't render correctly.
Comparing to another animated WebP library (YYImage), it seems that the decoder for WebP to get per frame images works wrong.
Also, current implementation does not support WebP disposal method,  which cause many animated WebP images look strange. And due to UIImage's API, the duration parameter just use the average duration for each frame, which is different from the animated WebP standard which allows set different duration per frame and also cause some images look strange.
For more detail and intuitive effect, you can go to the demo project to build and run.
Step:

find some animated WebP images' URLs.
create an FLAnimatedImageView or UIImageView instance and use sd_setImageWithURL: API to set image.
when showing animated WebP images, the images not showing correctly(looks strange).

By reading the decoder from YYImage and SDWebImage , I found that for single frame animated WebP, the libwebp's WebPBitstreamFeatures.height and width is the images actual size. But for multiple frame images, this will return the current frame canvas size(but not image size). This will make the UIImages array with the wrong size. We should useWebPDemuxGetI to get the actual size.
Looks at this animated webp images (actual size is 100x100), but the animated images array to use for UIImage -animatedImageWithImages: show the wrong size.

The YYImageCoder code reference :YYImage

There is an simple fix(which first create UIImage through CGImageCreate from WebPData. Then draw this UIImage to an new CGBitmapContext with actual size(not just the canvas size).
fix screenshot
Consider a higher performance(because create CGBitMapContext need resource), maybe we can modify the RGBA vector to do these).
The simple fix commit Fixed animated WebP image frame size wrong

Also, I found that the animated WebP dipose method(see here) is not support, currently is assume that all frames use the background disposal.
By sharing an blend canvas while decoding multiple frame animated webp image, we could do disposal and blend correctly(not relay on that last image).

PR is here: https://github.com/rs/SDWebImage/pull/1952

Because +[UIImage animatedImageWithImages:duration:] just use the average duration for each frame, which is not the animated WebP standard. Animated WebP can has different duration per frame. Therefore, we should divide the total duration and add image item multiple times to solve this issue. The code like this:
NSArray<UIImage *> *)images;
NSUInteger count = images.count;
int durations[count];
int const gcd = gcdArray(count, durations);
NSMutableArray<UIImage *> *animatedImages = [NSMutableArray arrayWithCapacity:count];
[images enumerateObjectsUsingBlock:^(UIImage * _Nonnull image, NSUInteger idx, BOOL * _Nonnull stop) {
    int duration = durations[idx];
    int repeatCount = duration / gcd;
    for (int i = 0; i < repeatCount; ++i) {
        [animatedImages addObject:image];
    }
}];

Fixed by #1952

authors: bpoplauschi dreampiggy

repo: rs/SDWebImage


text:
Remove locks in SDAnimatedImageView

New Pull Request Checklist

[x] I have read and understood the CONTRIBUTING guide

[x] I have read the Documentation

[x] I have searched for a similar pull request in the project and found none

[ ] I have updated this branch with the latest master to avoid conflicts (via merge from master or rebase)

[ ] I have added the required tests to prove the fix/feature I am adding

[ ] I have updated the documentation (if necessary)

[ ] I have run the tests and they pass

[ ] I have run the lint and it passes (pod lib lint)

This merge request fixes / reffers to the following issues: ...
Pull Request Description
Totally remove locks in SDAnimatedImageView. Currently we use lock to prevent thread race condition when operate frameBuffer. TBO, we can remove all these locks. We can see where we operate frameBuffer :

resetAnimatedImage, be called in setImage, so it would be called on main thread. (setImage is UIKit API, should call on main thread).
setImage, the same as above, should be called on main thread.
didReceiveMemoryWarning, be called on main thread.
displayDidRefresh, also be called on main thread.

So, the only place would produce race condition is when we prefetch frame in fetchQueue, but it's easy to fix, just use GCD to keep safe like below:
        NSOperation *operation = [NSBlockOperation blockOperationWithBlock:^{
            UIImage *frame = [animatedImage animatedImageFrameAtIndex:fetchFrameIndex];
            dispatch_async(dispatch_get_main_queue(), ^{
                self.frameBuffer[@(fetchFrameIndex)] = frame;
            });
        }];

Codecov Report

Merging #2473 into 5.x will increase coverage by 0.14%.
The diff coverage is 55%.

@@            Coverage Diff             @@
##              5.x    #2473      +/-   ##
==========================================
+ Coverage   77.87%   78.01%   +0.14%
==========================================
  Files          47       47
  Lines        6621     6610      -11
==========================================
+ Hits         5156     5157       +1
+ Misses       1465     1453      -12

Flag
Coverage Δ

#iOS
77.8% <55%> (ø)
:arrow_up:

#macOS
76.62% <55%> (-0.01%)
:arrow_down:

Impacted Files
Coverage Δ

SDWebImage/SDAnimatedImageView.m
70.19% <55%> (-0.3%)
:arrow_down:

SDWebImage/SDImageGIFCoder.m
87.79% <0%> (+0.39%)
:arrow_up:

SDWebImage/SDAnimatedImage.m
80.4% <0%> (+0.4%)
:arrow_up:

SDWebImage/SDWebImageDownloaderOperation.m
89.1% <0%> (+2.64%)
:arrow_up:

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 0f14258...aeca363. Read the comment docs.

Need some time to consider this issue. This not something only about thread-safe. It's about the animation rendering.
Why to use lock instead of GCD to dispatch, it's because lock can actually block main queue immediatelly until the fetch queue write the frame into the cache.
Consider this case : Current rendering frame is 0, and when next frame duration is reached, we need to render the 1 frame as fast as possible. Assume the thread-safe condition is reached (Query cache in main queue but fetchQueue was writing the 1 frame).
When using block, we will block main queue and let 1 frame sucessfully write to cache, then main queue get the 1 frame and render it.
However, if we use GCD to dispatch fetchQueue into main queue, the 1 frame will not be in the cache, so that 1 frame will be trigged as a cache miss. This time will not render the 1 frame. Even worse, this simple change of code will cause a issue that we re-fetch the 1 frame (Because the fetchFrame is also nil and start fetch 1 frame again)
So this need detailed consideration. I'll check it back later.

@dreampiggy 👌

When using block, we will block main queue and let 1 frame sucessfully write to cache, then main queue will be resumed, and get the 1 frame and render it.

🤔 Emm, have no idea where it would block to wait to cache?

Even worse, this simple change of code will cause a issue that we re-fetch the 1 frame (Because at this time, the fetchFrame is also nil and start fetch 1 frame again)

I think no re-fetch would be appear? fetchQueue only support one operation to run. self.fetchQueue.operationCount == 0

However, if we use GCD to dispatch fetchQueue into main queue, the 1 frame will not be in the cache until next runloop.

Change dispatch_async(dispatch_get_main_queue() to dispatch_sync(dispatch_get_main_queue(), I think it maybe the same as the original code.

@zhongwuzw > Emm, have no idea where it would block to wait to cache?
Because whether we decide to query the fetchFrame, is dependent on whether current frame is available just in time. So if we simply remove the LOCK and use GCD to write the frame cache, then we actually query the frame finished, we need to wait all of main queue job finished and write it into frame cache in the next runloop. It's late.

I think no re-fetch would be appear? fetchQueue only support one operation to run.
The fetch queue is just a operation queue. Why the code add operationCount == 0 because it base on the assumation that we can write to the cache in time. No need to query multiple frames. But however, using GCD to wait for main queue be operated, will make this assume failed. If we need to change it, we can do another solution, such as allow multiple query frame at the same time, but in the operation block add index guard to avoid different operation share the same index. But this need some more code to do.

Change dispatch_async(dispatch_get_main_queue() to dispatch_sync(dispatch_get_main_queue(), I think it maybe the same as the original code.
Actually this does not. Because dispatch_sync still need to submit your code into the last of main queue and wait for main queue to execute all previous code until this block been done. You just can ensure it blocks the operaion queue. However, when using block, the code will immediatelly (nearlly, actually) be executed, don't need to wait for main queue waiting code to be executed. So there are some little difference.

@dreampiggy

The fetch queue is just a operation queue. Why the code add operationCount == 0 because it base on the assumation that we can write to the cache in time, so one frame render, one check, one query is enough.

Nope, we can't do this assume, mostly, it would not in time. Let's assume most devices have 2 core, so only 2 jobs would be run at the same time, main thread, system, users, would do many works, most of the time, more than 2 threads, so thread context switch would not be prevented. NSOperationQueue's default priority is NSOperationQualityOfServiceBackground, priority is slow.

Actually this does not. Because dispatch_sync still need to submit your block code into the last of main queue and wait for main queue to execute all previous code until this block been done. Sync just ensure it blocks the operaion queue so that the operation is not been marked finished. However, when using block, it will interrupt the running main queue (what GCD can not)

TBO, I don't get what you want to express. I don't know wether you mess up the runloop, runloop is just a do while, a event loop, if no message, it sleep, when message becomes, it wake up and handle it, DisplayLink is just a timer, the interval it 1/60s, it added to runloop, so every 1/60s, runloop would handle it. So if we add block to main queue, no matter how many it is, if runloop is sleep, it would wake up the runloop, runloop would call all blocks one by one added before by __CFRUNLOOP_IS_SERVICING_THE_MAIN_DISPATCH_QUEUE__.

the following pair dispatch_semaphore_wait dispatch_semaphore_signal call does not cause performance lost. It just increase/decrease the semaphore count. Only when thread-safe race condition is apperared can block the calling queue.

Emm, 8 days ago, you said it would be the cost, see comment you leave. And not you said, cost is always exist, no matter wether or not thread-safe race condition, it would do atomic operation.

GCD will make a queue switch each time you access. And GCD's thread pool is limited to 64 on iOS. No need this extra cost.

Maybe you mean thread switch? thread switch is always exist, sync is just add a block to main queue.
GCD's thread pool is limited to 64, it's correct, and also what I want to say in Add autoreleasepool to release autorelease objects in advance when using GCD, in that PR, we can't do one operation to one queue. because one queue may spawn a new thread, so it's easy to make app hang, mean thread explosion, because user may use a number of threads, our framework also may use many threads, I don't remember which WWDC session, Apple developer give some demos how to use GCD and decode image, you can check it.
And, maybe this PR needs more people to review IMO, @bpoplauschi @skyline75489 @mythodeia .
I also would  check the SDAnimatedImageView to figure out wether it has any issue.

And for these two approaches. Today I try to run profile about these two approach. For most of cases, they behave some on performance. The lock there does not cause issue.
However, during the test, use simple GCD queue using the cid: aeca363ef169c5a5a5cb1bc0ab86bcf01184b12b, have a race ratio that can query the same frame index twice, even the memory is enough for full buffer. However, using lock does not contains this issue.

The demo I use, which contains the different test between SDAnimatedImageView && SDAnimatedImageView(With this PR's NO LOCK patch) && YYAnimatedImageView). You can try to run it as well. (run pod install after download, and you can also modify the SDImageGIFCoder code to check that assert)
SDWebImageTest.zip
Remove lock is OK. But I think, we should ensure this not cause some extra issue.
PS: Actually I don't want to spend more on it. If you can ensure this is OK, we can merge it now actually. Me && @bpoplauschi are hurried for 5.0.0 official release. I'm spending time to update documentation and tests, and check if there are anything important API break thing to do. Since this is a performance enhancement PR so we can also fix it in the furture (No API break). So maybe I'll put it into a low priority.

Guys, here is my opinion:

this improvement does not come based on a real problem found by our users or us
I would love to get rid of the locks, but it seems (at least to me) we do need them for safety
the improvements would be smalls
this case if different from the one we discussed about autoreleasepools, since that was based on actual measurements and with big impact
I simply suggest we don't merge this PR
@zhongwuzw hope you keep putting up PRs :) you're doing great. Just that some of them might not be merged. Same for everyone here.

@bpoplauschi 👌 I'll be go on to do some fixes. 😂 ❤
@dreampiggy I'm ok for this PR merge or not merge, no matter for me.
I see your ThreadSafeTest demo, it compares many locks performance, but for this situation, I want to do is free the main thread, because nearly all methods called on main thread, no need to use lock in these methods.
For the issue, I want to tell you, it's the issue of code reorder and thread context switch, not the issue of GCD.
Locks also has this issue.

authors: bpoplauschi codecov-io dreampiggy zhongwuzw


text:
Debian build seems to be broken?

We're trying to build a binary .deb package for libmbus so we can distribute that instead of statically linking it.  It would appear this build references files that no longer exist:
14-Jul-2020 16:07:00 | make[3]: Leaving directory '/mnt/src/obj-x86_64-linux-gnu'
-- | --
14-Jul-2020 16:07:00 | make -f CMakeFiles/libmbus.dir/build.make CMakeFiles/libmbus.dir/build
14-Jul-2020 16:07:00 | make[3]: Entering directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | make[3]: Nothing to be done for 'CMakeFiles/libmbus.dir/build'.
14-Jul-2020 16:07:00 | make[3]: Leaving directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | [100%] Built target libmbus
14-Jul-2020 16:07:00 | make[2]: Leaving directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | /usr/bin/cmake -E cmake_progress_start /mnt/src/obj-x86_64-linux-gnu/CMakeFiles 0
14-Jul-2020 16:07:00 | make -f CMakeFiles/Makefile2 preinstall
14-Jul-2020 16:07:00 | make[2]: Entering directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | make[2]: Nothing to be done for 'preinstall'.
14-Jul-2020 16:07:00 | make[2]: Leaving directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | Install the project...
14-Jul-2020 16:07:00 | /usr/bin/cmake -P cmake_install.cmake
14-Jul-2020 16:07:00 | -- Install configuration: "None"
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/share/pkgconfig/libmbus.pc
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/lib/x86_64-linux-gnu/libmbus.so
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/lib/x86_64-linux-gnu/cmake/libmbus/libmbusTargets.cmake
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/lib/x86_64-linux-gnu/cmake/libmbus/libmbusTargets-none.cmake
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/lib/x86_64-linux-gnu/cmake/libmbus/libmbusConfig.cmake
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/lib/x86_64-linux-gnu/cmake/libmbus/libmbusConfigVersion.cmake
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus/mbus-protocol-aux.h
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus/mbus-protocol.h
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus/mbus-serial.h
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus/mbus-tcp.h
14-Jul-2020 16:07:00 | -- Installing: /mnt/src/debian/tmp/usr/include/mbus/mbus.h
14-Jul-2020 16:07:00 | make[1]: Leaving directory '/mnt/src/obj-x86_64-linux-gnu'
14-Jul-2020 16:07:00 | dh_install
14-Jul-2020 16:07:00 | dh_install: Compatibility levels before 9 are deprecated (level 7 in use)
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/lib/lib*.a" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus-dev missing files: usr/lib/lib*.a
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/lib/lib*.so" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus-dev missing files: usr/lib/lib*.so
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/lib/*.la" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus-dev missing files: usr/lib/*.la
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/lib/pkgconfig/libmbus.pc" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus-dev missing files: usr/lib/pkgconfig/libmbus.pc
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/lib/lib*.so.*" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus1 missing files: usr/lib/lib*.so.*
14-Jul-2020 16:07:00 | dh_install: Cannot find (any matches for) "usr/bin/*" (tried in "." and "debian/tmp")
14-Jul-2020 16:07:00 | dh_install: libmbus1 missing files: usr/bin/*
14-Jul-2020 16:07:00 | dh_install: missing files, aborting
14-Jul-2020 16:07:00 | make: *** [binary] Error 2
14-Jul-2020 16:07:00 | debian/rules:6: recipe for target 'binary' failed
14-Jul-2020 16:07:00 | dpkg-buildpackage: error: debian/rules binary gave error exit status 2

Culprit seems to be this line: https://github.com/rscada/libmbus/blob/master/debian/libmbus-dev.install#L2
The build script we use used to work.  Is there something we missed when preparing the sources for building as a Debian package?

Sorry, for the inconvenience. During the conversion of the build system the target for the static library get dropped.
Unfortunately i'm not that cmake expert and partly regret the decision. Nevertheless i started my work, but it's still not usable. Maybe you have an idea what's the problem.

We had an issue where the .so file was named liblibmbus.so, I would assume that we removed one "lib" from the name.
So it is possible that the file that previously was called libmbus.a now instead is named mbus.a.
I don't know CMake and do not intend to learn it unless I really have to. So I am of little help.
I tried to build a .deb file, but my "employer" wanted the example programs as well, I was not able to solve that without starting to patch the build files, so I went with a normal .tar.gz where I could do things between the build/installation and the final packaging in an archive file,

The lib prefix issue has been fixed recently. But in case we don't have a solution at the end of the week, i think about reverting.

@sjlongland The debian directory is now deprecated. Please take a look at #175 and modify the build.sh to build the debian packages with cmake. I will add the binaries and doc later, after you gave me the okay this is going into right direction.

I don't think it because of the lib prefix.

@sjlongland The debian directory is now deprecated.

Are you certain. I think the the CMake/cpack could use some files from that directory. (But that is only my belife/assumtion.
I don't know cmake, nor deb packages nor cpack

I did build the Dockerfile.deb image. It seems that there is a static library being built, and it is presnt in the in the resulting -dev.deb file.
# ls -l /tmp/build/lib/libmbus.a
-rw-r--r-- 1 root root 325474 Jul 15 19:56 /tmp/build/lib/libmbus.a

# dpkg -c libmbus_0.9.0-dev.deb
...
-rw-r--r-- root/root    325474 2020-07-15 19:56 ./usr/lib/libmbus.a
...

@fredrik-sk Do you mean before or after my changes?

@lategoodbye : After #175

That works, although the build time was unexpectedly short.  A bigger issue is the file names don't conform to Debian packaging conventions.
Nominally these would be named libmbus_0.9.0_${ARCH}.deb and libmbus-dev_0.9.0_${ARCH}.deb (where ${ARCH} is the target architecture).  Instead, they're named libmbus_0.9.0-lib.deb and libmbus_0.9.0-dev.deb.  There's also no source Debian package produced via this method which makes re-building on other architectures easier.

Okay, was able to hack around the package naming:
        arch=$( dpkg --print-architecture )
        for f in *.deb; do
            pkgbasename=${f%_*}
            suffix=${f#${pkgbasename}_}
            version=${suffix%-*}
            case "$f" in
                *-lib.deb)
                    mv -v $f ${pkgbasename}_${version}_${arch}.deb
                    ;;
                *-dev.deb)
                    mv -v $f ${pkgbasename}-dev_${version}_${arch}.deb
                    ;;
                *)
                    mv -v $f ${f/.deb/_${arch}.deb}
                    ;;
            esac
        done

Until we start getting packages with files named *_amd64_amd64.deb, this will get us by I think. :-)

All those those debuild/debmake/cmake crap, drives me crazy :(

It can be a pain, but try asking your typical end user to do a ./configure && make && make install.
They just want to press a button and "magic happens", and it doesn't seem to matter whether it's a smartphone user pressing the install button on their phone's application repository, or whether its an engineer that wants to integrate M-Bus meters.

@sjlongland Can you please retest with build-deb.sh?

root@bc58b76794df:/tmp/work# ./build-deb.sh
Running autotools...
configure.ac:19: warning: LT_INIT was called before AM_PROG_AR
aclocal.m4:10027: AM_PROG_AR is expanded from...
configure.ac:19: the top level
…
libtoolize: and rerunning libtoolize and aclocal.
configure.ac:19: warning: LT_INIT was called before AM_PROG_AR
aclocal.m4:10027: AM_PROG_AR is expanded from...
configure.ac:19: the top level
configure.ac:19: warning: LT_INIT was called before AM_PROG_AR
aclocal.m4:10027: AM_PROG_AR is expanded from...
configure.ac:19: the top level
 dpkg-buildpackage -rfakeroot -us -uc -i -b
dpkg-buildpackage: warning: using a gain-root-command while being root
dpkg-buildpackage: info: source package libmbus
dpkg-buildpackage: info: source version 0.9.0
dpkg-buildpackage: info: source distribution xenial
dpkg-buildpackage: info: source changed by Stefan Wahren <stefan.wahren@i2se.com>
 dpkg-source -i --before-build work
dpkg-buildpackage: info: host architecture amd64
 fakeroot debian/rules clean
dh --with autoreconf clean
dh: Compatibility levels before 9 are deprecated (level 7 in use)
…
configure: creating ./config.status
config.status: creating Makefile
config.status: creating mbus/Makefile
config.status: creating test/Makefile
config.status: creating bin/Makefile
config.status: creating libmbus.pc
config.status: creating config.h
config.status: executing depfiles commands
config.status: executing libtool commands
configure: WARNING: unrecognized options: --disable-maintainer-mode
----------------------------------------------------------
Configuration:

        Source location:        .
        Compile:                gcc
        Compiler flags:         -g -O2
        Linker flags:            -version-info 0:9:0
        Host system type:       x86_64-pc-linux-gnu
        Install path:           /usr

        See config.h for further configuration.
----------------------------------------------------------
   dh_auto_build
dh_auto_build: Compatibility levels before 9 are deprecated (level 7 in use)
        make -j1
make[1]: Entering directory '/tmp/work'
make  all-recursive
make[2]: Entering directory '/tmp/work'
Making all in mbus
make[3]: Entering directory '/tmp/work/mbus'
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I..   -g -O2 -c -o mbus.lo mbus.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus.c  -fPIC -DPIC -o .libs/mbus.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus.c -o mbus.o >/dev/null 2>&1
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I..   -g -O2 -c -o mbus-protocol.lo mbus-protocol.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-protocol.c  -fPIC -DPIC -o .libs/mbus-protocol.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-protocol.c -o mbus-protocol.o >/dev/null 2>&1
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I..   -g -O2 -c -o mbus-tcp.lo mbus-tcp.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-tcp.c  -fPIC -DPIC -o .libs/mbus-tcp.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-tcp.c -o mbus-tcp.o >/dev/null 2>&1
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I..   -g -O2 -c -o mbus-serial.lo mbus-serial.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-serial.c  -fPIC -DPIC -o .libs/mbus-serial.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-serial.c -o mbus-serial.o >/dev/null 2>&1
/bin/bash ../libtool  --tag=CC   --mode=compile gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I..   -g -O2 -c -o mbus-protocol-aux.lo mbus-protocol-aux.c
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-protocol-aux.c  -fPIC -DPIC -o .libs/mbus-protocol-aux.o
libtool: compile:  gcc -DHAVE_CONFIG_H -I. -I.. -I.. -I.. -g -O2 -c mbus-protocol-aux.c -o mbus-protocol-aux.o >/dev/null 2>&1
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2  -version-info 0:9:0 -o libmbus.la -rpath /usr/lib mbus.lo mbus-protocol.lo mbus-tcp.lo mbus-serial.lo mbus-protocol-aux.lo
libtool: link: gcc -shared  -fPIC -DPIC  .libs/mbus.o .libs/mbus-protocol.o .libs/mbus-tcp.o .libs/mbus-serial.o .libs/mbus-protocol-aux.o    -g -O2   -Wl,-soname -Wl,libmbus.so.0 -o .libs/libmbus.so.0.0.9
libtool: link: (cd ".libs" && rm -f "libmbus.so.0" && ln -s "libmbus.so.0.0.9" "libmbus.so.0")
libtool: link: (cd ".libs" && rm -f "libmbus.so" && ln -s "libmbus.so.0.0.9" "libmbus.so")
libtool: link: ar cru .libs/libmbus.a  mbus.o mbus-protocol.o mbus-tcp.o mbus-serial.o mbus-protocol-aux.o
ar: `u' modifier ignored since `D' is the default (see `U')
libtool: link: ranlib .libs/libmbus.a
libtool: link: ( cd ".libs" && rm -f "libmbus.la" && ln -s "../libmbus.la" "libmbus.la" )
make[3]: Leaving directory '/tmp/work/mbus'
Making all in bin
make[3]: Entering directory '/tmp/work/bin'
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-scan.o mbus-tcp-scan.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-scan mbus-tcp-scan.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-scan mbus-tcp-scan.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-request-data.o mbus-tcp-request-data.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-request-data mbus-tcp-request-data.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-request-data mbus-tcp-request-data.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-request-data-multi-reply.o mbus-tcp-request-data-multi-reply.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-request-data-multi-reply mbus-tcp-request-data-multi-reply.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-request-data-multi-reply mbus-tcp-request-data-multi-reply.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-select-secondary.o mbus-tcp-select-secondary.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-select-secondary mbus-tcp-select-secondary.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-select-secondary mbus-tcp-select-secondary.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-scan-secondary.o mbus-tcp-scan-secondary.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-scan-secondary mbus-tcp-scan-secondary.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-scan-secondary mbus-tcp-scan-secondary.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-scan.o mbus-serial-scan.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-scan mbus-serial-scan.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-scan mbus-serial-scan.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-request-data.o mbus-serial-request-data.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-request-data mbus-serial-request-data.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-request-data mbus-serial-request-data.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-request-data-multi-reply.o mbus-serial-request-data-multi-reply.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-request-data-multi-reply mbus-serial-request-data-multi-reply.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-request-data-multi-reply mbus-serial-request-data-multi-reply.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-select-secondary.o mbus-serial-select-secondary.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-select-secondary mbus-serial-select-secondary.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-select-secondary mbus-serial-select-secondary.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-scan-secondary.o mbus-serial-scan-secondary.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-scan-secondary mbus-serial-scan-secondary.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-scan-secondary mbus-serial-scan-secondary.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-switch-baudrate.o mbus-serial-switch-baudrate.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-switch-baudrate mbus-serial-switch-baudrate.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-switch-baudrate mbus-serial-switch-baudrate.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-raw-send.o mbus-tcp-raw-send.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-raw-send mbus-tcp-raw-send.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-raw-send mbus-tcp-raw-send.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-tcp-application-reset.o mbus-tcp-application-reset.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-tcp-application-reset mbus-tcp-application-reset.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-tcp-application-reset mbus-tcp-application-reset.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
gcc -DHAVE_CONFIG_H -I. -I..  -I.. -I.. -I../src   -g -O2 -c -o mbus-serial-set-address.o mbus-serial-set-address.c
/bin/bash ../libtool  --tag=CC   --mode=link gcc  -g -O2 -L../mbus -version-info 0:9:0 -o mbus-serial-set-address mbus-serial-set-address.o -lmbus -lm
libtool: warning: '-version-info' is ignored for programs
libtool: link: gcc -g -O2 -o .libs/mbus-serial-set-address mbus-serial-set-address.o  -L../mbus /tmp/work/mbus/.libs/libmbus.so -lm
pod2man --release=0.9.0 --center=libmbus libmbus.pod \
        >.pod2man.tmp.$$ 2>/dev/null && mv -f .pod2man.tmp.$$ libmbus.1 || true
make[3]: Leaving directory '/tmp/work/bin'
make[3]: Entering directory '/tmp/work'
make[3]: Leaving directory '/tmp/work'
make[2]: Leaving directory '/tmp/work'
make[1]: Leaving directory '/tmp/work'
   dh_auto_test
dh_auto_test: Compatibility levels before 9 are deprecated (level 7 in use)
        make -j1 check VERBOSE=1
make[1]: Entering directory '/tmp/work'
Making check in mbus
make[2]: Entering directory '/tmp/work/mbus'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/tmp/work/mbus'
Making check in bin
make[2]: Entering directory '/tmp/work/bin'
make[2]: Nothing to be done for 'check'.
make[2]: Leaving directory '/tmp/work/bin'
make[2]: Entering directory '/tmp/work'
make[2]: Leaving directory '/tmp/work'
make[1]: Leaving directory '/tmp/work'
   create-stamp debian/debhelper-build-stamp
 fakeroot debian/rules binary
dh --with autoreconf binary
dh: Compatibility levels before 9 are deprecated (level 7 in use)
   dh_testroot
   dh_prep
   dh_installdirs
dh_installdirs: Compatibility levels before 9 are deprecated (level 7 in use)
   dh_auto_install
dh_auto_install: Compatibility levels before 9 are deprecated (level 7 in use)
        make -j1 install DESTDIR=/tmp/work/debian/tmp AM_UPDATE_INFO_DIR=no
make[1]: Entering directory '/tmp/work'
Making install in mbus
make[2]: Entering directory '/tmp/work/mbus'
make[3]: Entering directory '/tmp/work/mbus'
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/lib'
 /bin/bash ../libtool   --mode=install /usr/bin/install -c   libmbus.la '/tmp/work/debian/tmp/usr/lib'
libtool: install: /usr/bin/install -c .libs/libmbus.so.0.0.9 /tmp/work/debian/tmp/usr/lib/libmbus.so.0.0.9
libtool: install: (cd /tmp/work/debian/tmp/usr/lib && { ln -s -f libmbus.so.0.0.9 libmbus.so.0 || { rm -f libmbus.so.0 && ln -s libmbus.so.0.0.9 libmbus.so.0; }; })
libtool: install: (cd /tmp/work/debian/tmp/usr/lib && { ln -s -f libmbus.so.0.0.9 libmbus.so || { rm -f libmbus.so && ln -s libmbus.so.0.0.9 libmbus.so; }; })
libtool: install: /usr/bin/install -c .libs/libmbus.lai /tmp/work/debian/tmp/usr/lib/libmbus.la
libtool: install: /usr/bin/install -c .libs/libmbus.a /tmp/work/debian/tmp/usr/lib/libmbus.a
libtool: install: chmod 644 /tmp/work/debian/tmp/usr/lib/libmbus.a
libtool: install: ranlib /tmp/work/debian/tmp/usr/lib/libmbus.a
libtool: warning: remember to run 'libtool --finish /usr/lib'
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/include/mbus'
 /usr/bin/install -c -m 644 mbus.h mbus-protocol.h mbus-tcp.h mbus-serial.h mbus-protocol-aux.h '/tmp/work/debian/tmp/usr/include/mbus'
make[3]: Leaving directory '/tmp/work/mbus'
make[2]: Leaving directory '/tmp/work/mbus'
Making install in bin
make[2]: Entering directory '/tmp/work/bin'
make[3]: Entering directory '/tmp/work/bin'
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/bin'
  /bin/bash ../libtool   --mode=install /usr/bin/install -c mbus-tcp-scan mbus-tcp-request-data mbus-tcp-request-data-multi-reply mbus-tcp-select-secondary mbus-tcp-scan-secondary mbus-serial-scan mbus-serial-request-data mbus-serial-request-data-multi-reply mbus-serial-select-secondary mbus-serial-scan-secondary mbus-serial-switch-baudrate mbus-tcp-raw-send mbus-tcp-application-reset mbus-serial-set-address '/tmp/work/debian/tmp/usr/bin'
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-scan /tmp/work/debian/tmp/usr/bin/mbus-tcp-scan
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-request-data /tmp/work/debian/tmp/usr/bin/mbus-tcp-request-data
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-request-data-multi-reply /tmp/work/debian/tmp/usr/bin/mbus-tcp-request-data-multi-reply
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-select-secondary /tmp/work/debian/tmp/usr/bin/mbus-tcp-select-secondary
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-scan-secondary /tmp/work/debian/tmp/usr/bin/mbus-tcp-scan-secondary
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-scan /tmp/work/debian/tmp/usr/bin/mbus-serial-scan
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-request-data /tmp/work/debian/tmp/usr/bin/mbus-serial-request-data
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-request-data-multi-reply /tmp/work/debian/tmp/usr/bin/mbus-serial-request-data-multi-reply
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-select-secondary /tmp/work/debian/tmp/usr/bin/mbus-serial-select-secondary
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-scan-secondary /tmp/work/debian/tmp/usr/bin/mbus-serial-scan-secondary
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-switch-baudrate /tmp/work/debian/tmp/usr/bin/mbus-serial-switch-baudrate
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-raw-send /tmp/work/debian/tmp/usr/bin/mbus-tcp-raw-send
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-tcp-application-reset /tmp/work/debian/tmp/usr/bin/mbus-tcp-application-reset
libtool: warning: '/tmp/work/mbus/libmbus.la' has not been installed in '/usr/lib'
libtool: install: /usr/bin/install -c .libs/mbus-serial-set-address /tmp/work/debian/tmp/usr/bin/mbus-serial-set-address
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/share/man/man1'
 /usr/bin/install -c -m 644 libmbus.1 mbus-tcp-scan.1 mbus-tcp-request-data.1 mbus-tcp-request-data-multi-reply.1 mbus-tcp-select-secondary.1 mbus-tcp-scan-secondary.1 mbus-tcp-raw-send.1 mbus-serial-scan.1 mbus-serial-request-data.1 mbus-serial-request-data-multi-reply.1 mbus-serial-select-secondary.1 mbus-serial-scan-secondary.1 mbus-serial-switch-baudrate.1 '/tmp/work/debian/tmp/usr/share/man/man1'
make[3]: Leaving directory '/tmp/work/bin'
make[2]: Leaving directory '/tmp/work/bin'
make[2]: Entering directory '/tmp/work'
make[3]: Entering directory '/tmp/work'
make[3]: Nothing to be done for 'install-exec-am'.
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/share/doc/libmbus-0.9.0'
 /usr/bin/install -c -m 644 README.md COPYING hardware/MBus_USB.pdf hardware/MBus_USB.txt '/tmp/work/debian/tmp/usr/share/doc/libmbus-0.9.0'
 /bin/mkdir -p '/tmp/work/debian/tmp/usr/lib/pkgconfig'
 /usr/bin/install -c -m 644 libmbus.pc '/tmp/work/debian/tmp/usr/lib/pkgconfig'
make[3]: Leaving directory '/tmp/work'
make[2]: Leaving directory '/tmp/work'
make[1]: Leaving directory '/tmp/work'
   dh_install
dh_install: Compatibility levels before 9 are deprecated (level 7 in use)
   dh_installdocs
   dh_installchangelogs
   dh_installman
   dh_perl
   dh_link
   dh_strip_nondeterminism
   dh_compress
   dh_fixperms
   debian/rules override_dh_strip
make[1]: Entering directory '/tmp/work'
dh_strip --dbg-package=libmbus1-dbg
dh_strip: Compatibility levels before 9 are deprecated (level 7 in use)
make[1]: Leaving directory '/tmp/work'
   dh_makeshlibs
   dh_shlibdeps
dpkg-shlibdeps: warning: symbol pow used by debian/libmbus1/usr/lib/libmbus.so.0.0.9 found in none of the libraries
dpkg-shlibdeps: warning: package could avoid a useless dependency if debian/libmbus1/usr/bin/mbus-tcp-application-reset debian/libmbus1/usr/bin/mbus-tcp-request-data debian/libmbus1/usr/bin/mbus-tcp-raw-send debian/libmbus1/usr/bin/mbus-serial-scan debian/libmbus1/usr/bin/mbus-serial-scan-secondary debian/libmbus1/usr/bin/mbus-serial-select-secondary debian/libmbus1/usr/bin/mbus-tcp-scan-secondary debian/libmbus1/usr/bin/mbus-serial-request-data-multi-reply debian/libmbus1/usr/bin/mbus-serial-request-data debian/libmbus1/usr/bin/mbus-tcp-scan debian/libmbus1/usr/bin/mbus-serial-set-address debian/libmbus1/usr/bin/mbus-tcp-request-data-multi-reply debian/libmbus1/usr/bin/mbus-tcp-select-secondary debian/libmbus1/usr/bin/mbus-serial-switch-baudrate were not linked against libm.so.6 (they use none of the library's symbols)
   dh_installdeb
dh_installdeb: Compatibility levels before 9 are deprecated (level 7 in use)
   dh_gencontrol
   dh_md5sums
   dh_builddeb
dpkg-deb: building package 'libmbus-dev' in '../libmbus-dev_0.9.0_amd64.deb'.
dpkg-deb: building package 'libmbus1-dbg' in '../libmbus1-dbg_0.9.0_amd64.deb'.
dpkg-deb: building package 'libmbus1' in '../libmbus1_0.9.0_amd64.deb'.
 dpkg-genbuildinfo --build=binary
 dpkg-genchanges --build=binary >../libmbus_0.9.0_amd64.changes
dpkg-genchanges: info: binary-only upload (no source code included)
 dpkg-source -i --after-build work
dpkg-buildpackage: info: binary-only upload (no source included)
Now running lintian...
warning: the authors of lintian do not recommend running it with root privileges!
E: libmbus changes: bad-distribution-in-changes-file xenial
W: libmbus1: package-name-doesnt-match-sonames libmbus0
W: libmbus1: extra-license-file usr/share/doc/libmbus-0.9.0/COPYING
W: libmbus1: manpage-has-bad-whatis-entry usr/share/man/man1/libmbus.1.gz
W: libmbus1: binary-without-manpage usr/bin/mbus-serial-set-address
W: libmbus1: binary-without-manpage usr/bin/mbus-tcp-application-reset
Finished running lintian.
root@bc58b76794df:/tmp/work#

Looking good!  That was on Debian Stretch (v9) via Docker.

Have a look at https://github.com/rscada/libmbus/pull/178 and see what you think.  This adds in source packages, and also allows for signing of packages, should a user require it.

authors: fredrik-sk lategoodbye sjlongland

repo: rscada/libmbus


text:
Repo build issues

From @kong0329:

Hi Raj,
I have questions about magma_cgra.

When I run python cb/cb_wrapper_main.py test_cb/cb.vp command, an error occurs.

  File "cb/cb_wrapper_main.py", line 6, in <module>
    from cb.cb_wrapper import define_cb_wrapper
ModuleNotFoundError: No module named 'cb.cb_wrapper'; 'cb' is not a package```

I suspected that same name of package name(cb) and module name(cb.py) so removed cb.py and tried again.
Now I get this error.

  File "cb/cb_wrapper_main.py", line 6, in <module>
    from cb.cb_wrapper import define_cb_wrapper
ModuleNotFoundError: No module named 'cb'```

Could you let me know how to resolve it?

This is to make sure what I understood about magma_cgra is correct.
<Let's say we design new module X>
a. Implement X.vp (Genesis generator file)
b. Implement wrapper. X_wrapper.py and X_wrapper_main.py
c. Implement magma file. X.py
d. Implement functional file. X_functional.py
e. Implement testbench. X_tb.v
f1. test_wrapper - test if wrapper works.
f2. test_functional - test if functional model is correct
f3. test_regression - test if generator magma and genesis behave same. test both behave as functional model.

Is this correct?
In the future, can we just skip a, b, f1 if we only stick to magma, not genesis?

Lastly, the point of magma is to increase the abstraction layer to python(magma), right?
If so, how is testbench file X_tb.v generated automatically according to parameters?
If we still have to implement testbench file in verilog, it would be inefficient.

Thanks for having great system Raj.

From @rsetaluri:
Hi Taeyoung, good questions
raj [5:19 PM]

you do not need to change around the file names for this. checkout the Usage section of the readme. particularly

pytest                           # install with pip install pytest```

actually, you will also need to do `pip install -e .` to run not just the tests but also `cb_wrapper_main.py`
Let me know if that works

2) Yes this is mostly correct. We are actually changing around some of the file names. This issue — https://github.com/rsetaluri/magma_cgra/issues/12 -- has what we want to do going forward regarding naming conventions. See the `mem` directory to see an example fo this
Yes, without Genesis we do not need those steps. But we do not think that we will move away from verilog entirely soon (edited)
3) Yes that is the intended use of magma. The testbench file `X_tb.v` is not necessarily a fundamental piece of this and would not generally be automatically generated. In this example we’ve included it just to show that you can also hook up with other testing paradigms. Especially hardware people who are already familiar with system verilog test benches can also use our system.
However, we are working to create a layer of abstraction on top of testing — you are right, it is not convenient to write out tests by hand. We are working on a high level system which you can pass in a functional model or test vectors and then automatically generate a set of tests which can be run against genesis version, magma version, etc.
For now you do not need to write a verilog based test bench (you can if you would like however). For now we are focused on adding functional models + wrappers + tests following the format of the cb module. We will slowly add magma versions
Does this make sense?

From @kong0329:
Hi Raj,
Thanks for your answers. They really helped a lot.
I still have some follow-up questions, so I appreciate if you could answer them.

This is clear. Thanks a lot.

If you slowly add magma, then until you have magma, the main goal of this system is to have functional model in python and easily test it with high level python, right?

For this, I am still facing some issues, and I think all these issues are just setup issues, which is annoying haha.
• pytest test_cb/test_regression.py
When I run this command, test fails in kiwi.
I found that it is related to coreir library path.
Even though I installed coreir package by running python setup.py install, running pytest couldn't find the library path, so I had to run below commands.

wget https://github.com/rdaly525/coreir/releases/download/v0.0.12/coreir.tar.gz
mkdir coreir_release;
tar -xf coreir.tar.gz -C coreir_release --strip-components 1;
export PATH=~/coreir_release/bin:$PATH;
export LD_LIBRARY_PATH=~/coreir_release/lib:$LD_LIBRARY_PATH;

Is there another way other than adding coreir library path to .bashrc?
• pytest test_cb/test_regression_ncsim.py
When I run this command, test aborted with below output.
test_cb/test_regression_ncsim.py Aborted (core dumped)

I further investigated it, and found compile_to_verilog function fails to open cached json file, but I am not sure.
Since ncsim.py is not tested in travis machine, I wonder whether it is working in any local machine.
Could you let me know how to solve it?
• python cb/cb_wrapper_main.py test_cb/cb.vp
This command doesn't work for me. Could you check whether above command works?
If cb.py file is in cb folder, cb_wrapper_main.py regards cb in import cb.cb_wrapper as module, not package.
Many thanks, Raj!
Thanks

@kong0329 I'm assuming you're running on kiwi.
I was able to reproduce the issue with test_cb/test_regression_ncsim.py Aborted (core dumped), it's because we're using /tmp on a shared machine and the file it tried to create already existed, but was owned by a different user so we didn't have permission to edit it.
The commit in the branch compile-with-tempdir should fix it, you can try it out with git checkout compile-with-tempdir, or wait until it's merged into mater.

@kong0329 for python cb/cb_wrapper_main.py test_cb/cb.vp, if you're on the latest master this won't work since the files have been renamed/reorganized. Can you try python cb/cb_genesis2.py cb/genesis/cb.vp? Also, if you can include the error log, that would be helpful.

You can install coreir without editing your path by installing it directly into your system path, see
https://github.com/rsetaluri/magma_cgra/pull/26/files for example instructions. The release comes with a makefile that allows you to use make install

Closing. See pr #25 for @kong0329 's approval.

authors: leonardt rsetaluri


text:
Update index.mdx

Corrected the link ' RSocket protocol ' to https://rsocket.io/about/protocol/ as it led to NOT FOUND (in Guides section).
Motivation:
To correct the guide.
Modifications:
Corrected the link ' RSocket protocol ' to https://rsocket.io/about/protocol/ as it led to NOT FOUND (in Guides section).
Result:
Link works :)

Thanks for the fix!

authors: AndriiMaliuta viglucci

repo: rsocket/rsocket-website