text:
VS Code integration

Visual Studio Code is a really nice development environment, even someone such as myself who has stuck with vim for many years is considering switching over to it.
It would be cool if we could provide some documentation and/or VS Code extensions to make VS Code more zephyr-aware. This ticket can be a catch-all for integration tasks/ideas. So far I can think of:

Best way to get tab settings correct across different file types in Zephyr
Somehow populate header information to provide those definitions only provided by Kconfig, so we can correctly see what preprocessor directives are active, right now they are all greyed out for CONFIG_* variables. Probably need to be able to set what board/project configuration we are building for.
Integration for interacting with west
Integration for running CMake and doing builds for a particular project directory
Interactive debugging to connected hardware
Integration to run an emulator (and interactively debug it if desired) for QEMU or other emulation targets
Some kinds of lintian checks can be shown directly in the editor GUI - wonder if we could integrate checkpatch into this

Other ideas, please comment!

For Kconfig, it's because autoconf.h is included automatically:
https://github.com/zephyrproject-rtos/zephyr/blob/9d5d2528947b512d85c212decaff7c1822d2e69d/CMakeLists.txt#L214
Would need to either include it manually where it's needed or tell VS Code about it somehow.

cmake/ide/eclipse_cdt4_generator_amendment.cmake seems to already have some stuff related to autoconf.h.

Would need to either include it manually where it's needed or tell VS Code about it somehow.

We'd need to generate it too, since it's a build artifact that depends on the selected board and defconfig.
Same deal with DTS #defines

On Kconfig: https://github.com/luveti/kconfig-vscode

Another nice improvement would be to enable search across modules, which I haven't been able to get working so far (while it is somewhat documented).

* Best way to get tab settings correct across different file types in Zephyr

This should already be supported using the .editorconfig file we have in the tree. That works for me. Alternativly, you could set this directly:
https://code.visualstudio.com/docs/getstarted/settings#_language-specific-editor-settings

There is a closed issue https://github.com/zephyrproject-rtos/zephyr/issues/17645 with some configs for vscode. Maybe it helps.

I would also add that some default configuration for https://marketplace.visualstudio.com/items?itemName=marus25.cortex-debug would be helpful for the people developing with ARM architecture.
I'm pretty sure this is what platform.io uses with some improvements https://platformio.org/platformio-ide
So having some support for

OS level awareness for debugging in VS code is also an improvement.

CC: @trond-snekvik  Maybe you have some input here?

I actually made a couple of extensions that I'm not sure are quite ready for prime time yet, but they're working quite well for my day to day stuff:

vscode-DeviceTree adds syntax highlighting, type checking, code completion, go to defintion and syntax checking to deviceTree files, written specifically for the Zephyr RTOS.
vscode-Kconfig adds highlighting, code go to definition and completion for Kconfig and .conf files, as well as type checking, basic linting and syntax validation for .conf files. It has some significant performance issues still (it reparses the entire kconfig tree on every change), so use at your own risk :)
vscode-doxygen-generator is a generic Doxygen header API generator for C. It improves syntax highlighting for doxygen tags, and adds a snippet generator for doxygen comments (Place your cursor on a function definition, and press Ctrl+Q).

The extensions can be installed by cloning them under ~/.vscode/extensions, and calling npm install . from inside the folder (let me know if there are any issues with this, as I've just installed this on my personal computers).
As mentioned, I've only used them for my personal setup, and they're not quite ready for public usage yet. They're all kind of personal pet projects, so they come with no guarantees. I'd like to make packages out of them and publish them to the VS Code extension repo eventually, so if anyone would like to try them out and provide some feedback, that would speed up the process!
They're not very intrusive extensions (apart from the syntax highlighting, I suppose), so I think they're fairly safe to install :)

Here is a helpful setting in my workspace c_cpp_properties.json:
"compileCommands": "~/Documents/source/blinky/build/compile_commands.json"

I also found some helpful information here:
https://gitlab.endian.se/simon/vscode-zephyr-intro
In particular, the forced includes and defines to get IntelliSense to work properly:
"forcedInclude": [
                "~/Documents/source/blinky/build/zephyr/include/generated/autoconf.h",
                "~/Documents/source/zephyrproject/zephyr/include/toolchain/zephyr_stdint.h",
                "~/Documents/source/blinky/.vscode/c_cpp_undef.h"
],
"defines": [
                "BUILD_VERSION=v2.1.0-rc1-141-g3cc5bda2fa10",
                "KERNEL",
                "_FORTIFY_SOURCE=2",
                "__PROGRAM_START",
                "__ZEPHYR__=1"
],

Also add this to your CMakeLists.txt for your project:
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

CMake can be configured to generate a compile_commands.json.  However, the file is for the application which leads to problems when you open a component of the Zephyr RTOS into vscode.  For instance, if I open drivers/i2c/i2c_sam0.c I will see an error that there is no target for that file.  This is true since the compilation of the OS is controlled by other CMake processes.  Vscode opens the file and parses using the include paths and forced includes and defines specified in the config file.
What would be useful here is to be able to generate the compile_commands.json file for all the other Zephyr RTOS related files.  That way, vscode would be able to open and parse any of the OS files (drivers, kernel, subsystems, modules, etc).  At the moment, I have to rely on the Include paths and forced includes and defines in my config.  These could be eliminated all together if I could get the compile_commands.json for all the components of Zephyr.

Here is an example of a task I use to invoke west to do a build:
{
	"label": "west build auto",
	"type": "shell",
	"group": "build",
	"command": "west",
	"args": [
	    "build",
	    "-p",
	    "auto",
	    "-b",
	    "scanimetrics_witap_isb",
	    "${workspaceFolder}"
	],
	"options": {
	    "cwd": "${workspaceFolder}",
	    "env": {
		"ZEPHYR_BASE": "/Users/stevenslupsky/Documents/source/zephyrproject/zephyr",
		"ZEPHYR_TOOLCHAIN_VARIANT": "gnuarmemb",
		"GNUARMEMB_TOOLCHAIN_PATH": "/usr/local/",
		"VS_CODE_BOARD_TO_BUILD": "scanimetrics_witap_isb",
		"VS_CODE_ZEPHYR_BIN_NAME": "zephyr.elf",
	    }
	},
	"dependsOn": [],
	"problemMatcher": [ "$gcc" ]
}

I found another way to integrate Zephyr with vscode using the CMake-tools extension.  I had been playing around with this extension a while back but really did not understand how to use it.  The key to use it for Zephyr is the "configurationProvider" property in c_cpp_properties.json.
Install the vector-of-bool (now microsoft) Cmake-tools extension.  Then, edit your c_cpp_properties.json in your project's .vscode directory and add the following configuration:
{
	"name": "Cmake",
	"configurationProvider": "vector-of-bool.cmake-tools"
}

Select the "Cmake" configuration that was added by clicking the configuration button in the bottom right portion of the status bar beside the "Tweet Feedback" button (it is typically set to "Mac" or "Win").  When you select the "Cmake" configuration (or whatever name you gave it), this automatically finds compile_commands.json according to the CMake extension settings (you can change the default if you need to).
To get the Zephyr components to parse properly, edit the CMakeLists.txt files in each of the Zephyr directories that contains source files you want to be able to open with your project.  Add the following line to each file:
set(CMAKE_EXPORT_COMPILE_COMMANDS ON)

In my case, I edited CMakeLists.txt in the following directories:
arch/arm
boards
drivers
kernel
soc
subsys
Build your project.  Check your compile_commands.json file and now you will see a number of additional entries for all the Zephyr components.
Now, when you open your Zephyr source files, they should be parsed properly by the c_cpp extension.

Some other useful extensions:
ARM, Dan Underwood
CMake Tools, Microsoft
Cortex-Debug, marus25
DeviceTree, Pietro Lorefice
Doxygen Documentation Generator, Christoph Schlosser
Git Graph, mhutchie
Intel HEX format, keroc
kconfig, luveti

There is one problem with using the compile_commands.json workflow with VSCode.  When you apply the CMAKE_EXPORT_COMPILE_COMMANDS setting, it doesn't seem to propagate to all the required files.
For instance, sam0_rtc_timer.c does not appear in my compile_commands.json.  I have not determined why it is not being included.

@SebastianBoe Any idea why the compile commands would not be consistent with the ninja build file?

I couldn't say why, but it seems that setting the variable from CMakeLists.txt is unofficially not supported. I would try setting it from CLI to ensure it is always set as a cache variable.
https://gitlab.kitware.com/cmake/cmake/issues/16588

@sslupsky Given the answer from Sebastian, try setting the compile command as was explained here:
https://github.com/zephyrproject-rtos/zephyr/issues/21119#issuecomment-578282316

Awesome discussion all around! I can't help but thinking that we can capture this with some automation via a new VS Code extension. If some of the scaffolding and configuration was implemented in west (like task generation) and the extension wrapped west, installed other extensions as deps - you could have a very nice developer experience out of the box. We did something similar when I was at Particle. Anyone with JS experience interested in collaborating? I know their APIs pretty well but I'm much more a Product guy than a good developer :)

@joerchan Thank you for the tip.  I can confirm that setting the CMAKE_EXPORT_COMPILE_COMMANDS from the command line works better than using the setting in the CMakeLists.txt file.  Now all the zephyr components are included in the compile_commands.json file.  Hooray!  :-)
One problem remains though.  Some of the include paths do not resolve properly.  For instance, #include <soc.h> comes up with a list of 76 definitions.  This prevents anything that includes soc.h from linting properly with intellisense.  Unfortunately, soc.h is quite a popular file to include ;-).
I have raised an issue with the VSCode team about this problem since it appears to be specific to VSCode at the moment:
https://github.com/microsoft/vscode-cmake-tools/issues/1120
Has anyone here been able to resolve this?

For clarification for anyone reading this, I am using the Multi-root workspace in VSCode.  This allows me to configure VSCode to work with projects that have multiple "sub projects".  In my use case, I have root folders for:
application (project)
custom board
zephyr tree (base)
zephyr modules
The multi root capability of VSCode is really quite helpful for Zephyr projects.  Here is a copy of the .code-workspace file:
{
	"folders": [
		{
			"path": "isb",
			"name": "app"
		},
		{
			"path": "zephyrproject/zephyr/boards/arm/scanimetrics_witap_isb",
			"name": "board"
		},
		{
			"path": "zephyrproject/zephyr",
			"name": "zephyr"
		},
		{
			"path": "zephyrproject/modules",
			"name": "modules"
		}
	],
	"settings": {
		"C_Cpp.default.includePath": [
			"${workspaceFolder:zephyr}/include",
			"${workspaceFolder:zephyr}/include/drivers",
			"${workspaceFolder:zephyr}/soc/arm/atmel_sam0/samd21",
			"${workspaceFolder:zephyr}/soc/arm/atmel_sam0/common/.",
			"${workspaceFolder:zephyr}/drivers",
			"${workspaceFolder:zephyr}/ext/lib/fnmatch/.",
			"${workspaceFolder:zephyr}/ext/hal/cmsis/Core/Include",
			"${workspaceFolder:zephyr}/subsys/usb",
			"${workspaceFolder:modules}/hal/atmel/asf/sam0/include/samd21",
			"${workspaceFolder:modules}/fs/littlefs",
			"${workspaceFolder:modules}/fs/fatfs/include",
			"${workspaceFolder:modules}/fs/nffs/include/nfss",
			"${workspaceFolder:zephyr}/lib/libc/minimal/include",
			"/usr/local/Cellar/arm-none-eabi-gcc/8-2018-q4-major/gcc/bin/../lib/gcc/arm-none-eabi/8.2.1/include",
			"/usr/local/Cellar/arm-none-eabi-gcc/8-2018-q4-major/gcc/bin/../lib/gcc/arm-none-eabi/8.2.1/include-fixed",
			"${workspaceFolder:app}/build/zephyr/include/generated"
		],
		"C_Cpp.default.forcedInclude": [
			"${workspaceFolder:app}/build/zephyr/include/generated/autoconf.h",
			"${workspaceFolder:zephyr}/include/toolchain/zephyr_stdint.h",
			"${workspaceFolder:app}/.vscode/c_cpp_undef.h"
		],
		"C_Cpp.default.defines": [
			"BUILD_VERSION=v2.1.0-rc1-141-g3cc5bda2fa10",
			"KERNEL",
			"_FORTIFY_SOURCE=2",
			"__PROGRAM_START",
			"__ZEPHYR__=1"
		],
		"C_Cpp.default.compilerPath": "/usr/local/bin/arm-none-eabi-gcc",
		"C_Cpp.default.cStandard": "c11",
		"C_Cpp.default.cppStandard": "c++17",
		"C_Cpp.default.intelliSenseMode": "gcc-x64",
		"C_Cpp.default.compileCommands": "${workspaceFolder:app}/build/compile_commands.json",
		"files.associations": {
			"zephyr.h": "c",
			"kernel.h": "c",
			"kernel_includes.h": "c",
			"log.h": "c",
			"printk.h": "c",
			"littlefs.h": "c",
			"i2c.h": "c",
			"usb_device.h": "c",
			"soc.h": "c",
			"samd21g18a.h": "c",
			"power.h": "c",
			"watchdog.h": "c",
			"i2c_ll_stm32.h": "c",
			"init.h": "c",
			"device.h": "c",
			"cpu.h": "c",
			"generated_dts_board.h": "c",
			"system_timer.h": "c",
			"clock_control.h": "c",
			"sys_clock.h": "c",
			"dma.h": "c",
			"hwinfo.h": "c",
			"gpio.h": "c",
			"errno.h": "c",
			"stdio.h": "c",
			"version.h": "c"
		},
		"terminal.integrated.env.osx": {
			"ZEPHYR_BASE": "${workspaceFolder:zephyr}/",
			"ZEPHYR_TOOLCHAIN_VARIANT": "gnuarmemb",
			"GNUARMEMB_TOOLCHAIN_PATH": "/usr/local/",
			"PATH": "$ZEPHYR_BASE/scripts:${env:PATH}"
		},
		"editor.tabSize": 8,
		"editor.insertSpaces": false
	}
}

@SebastianBoe Thank you, the command line option resolved my issue with the compile_commands.json file not including all the zephyr targets.

@beriberikix That is a great idea ... that would simplify the setup process enormously by eliminating the need for someone to learn about the details of configuring VSCode.  I have grown to admire VSCode tremendously and it keeps getting better.  I have one nagging issue related to git that I haven't been able to figure out.  Seems like VSCode will occasionally max out the CPU in one of its' threads.  The problem seems related to staging selected code changes but I am not completely certain that is the cause; rather it is likely just a coincidence.  VSCode has a helpful command when this happens, "Developer: reload window".  This is a lifesaver.  You loose your undo history but pretty much everything else comes back to where you were minus the laggy responsiveness due to the high cpu usage.

I published the Kconfig extension on the marketplace last week. It provides full language support for all Kconfig and .conf files out of the box, including code completion, go to definition, syntax highlighting and linting. It should improve the VS Code experience for Kconfig quite a bit for both application developers and stack maintainers.
The extension depends on West v0.7.0 or newer. Older versions should work, but there may be some manual configuration required. Feel free to contact me on slack if you have any questions, issues or feature requests :)

@trond-snekvik Thanks for this! I've been using it for the last few days and it's great. Hoping to see more zephyr support for VSCode in the future.

Just stumbled on this document by segger on how to integrate J-link debugging with vscode. It's not zephyr specific, but the general setup is applicable to zephyr as well, so posting here for reference.
https://wiki.segger.com/J-Link_Visual_Studio_Code

I have published the Devicetree extension now as well.
Before publishing, I rewrote most of it to provide full language support for all Devicetree files, and process the C preprocessor symbols, allowing overlays, board files and shields to be included in the processing. It has quite a bit more features than the kconfig extension, but should be less intrusive.
Again, feel free to contact me for feature requests and bug reports ☺

Someone should create a Zephyr extension pack! Perhaps it can be officially managed by a maintainer :)

Someone should create a Zephyr extension pack! Perhaps it can be officially managed by a maintainer :)

should def do this - so those of us in vim can benefit from it in Coq.

I've had some good luck using VS Code and its support for Language Server to do intellisense stuffs by having CMake generate a compilation database (that VS Code then consumes automatically, assuming the build directory is in the project).
The CMake parameter is CMAKE_EXPORT_COMPILE_COMMANDS, and here's examples of building an app:
west build -b <board> <app directory> -- -DCMAKE_EXPORT_COMPILE_COMMANDS=1

If you always want to generate the compilation database, you can also do:
west config build.cmake-args -- -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

Hi does this automatically works if I have /build dir on my vs code workspace?
any further config I need to do to make the IntelliSense stuff works?

I've had some good luck using VS Code and its support for Language Server to do intellisense stuffs by having CMake generate a compilation database (that VS Code then consumes automatically, assuming the build directory is in the project).
The CMake parameter is CMAKE_EXPORT_COMPILE_COMMANDS, and here's examples of building an app:
west build -b <board> <app directory> -- -DCMAKE_EXPORT_COMPILE_COMMANDS=1

If you always want to generate the compilation database, you can also do:
west config build.cmake-args -- -DCMAKE_EXPORT_COMPILE_COMMANDS=ON

Hi does this automatically works if I have /build dir on my vs code workspace?
any further config I need to do to make the IntelliSense stuff works?

Hi, v0.12.0 SDK open-ocd has RTOS awareness, verified by https://github.com/zephyrproject-rtos/openocd/issues/42#issue-788565903
The following RTOS values are supported by OpenOCD: eCos ThreadX FreeRTOS ChibiOS embKernel mqx uCOS-III auto

so what's the way to inform cortex-debug about zephyr awareness?

Hi, v0.12.0 SDK open-ocd has RTOS awareness, verified by https://github.com/zephyrproject-rtos/openocd/issues/42#issue-788565903
The following RTOS values are supported by OpenOCD: eCos ThreadX FreeRTOS ChibiOS embKernel mqx uCOS-III auto

so what's the way to inform cortex-debug about zephyr awareness?

I just watched the discussion about Zephyr Developer Environments.
In the discussion "Docker with Remote Plugin for VScode" was mentioned, that is what I use to setup
my development environment on Windows for zephyr.
I would like to share that setup.
Requirements:

Windows with WSL2 installed
Docker with WSL2 backend
Pyocd on Windows
Ozone installed on Windows

I start the container always from WSL2 (for example Ubuntu Distro),
because if I start it from a Windows path directly, I experienced a
significantly performance drop.
My config files for vscode:
.devcontainer/devcontainer.json
{
	"name": "Zephyr development environment",
    // I use a specific tag for the docker image, that vscode can detect an update from docker image
	"image": "docker.io/zephyrprojectrtos/zephyr-build:v0.17.5",
    // It is pretty nice that you can define the plugins you want
    // and it will be automaticly installed when you open the project.
	"extensions": [
		"ms-python.python",
		"ms-vscode.cpptools",
		"twxs.cmake",
		"marus25.cortex-debug",
		"trond-snekvik.devicetree",
		"trond-snekvik.kconfig-lang",
		"editorconfig.editorconfig",
		"jebbs.plantuml",
		"pkief.material-icon-theme",
		"eamodio.gitlens"
	],
	"remoteUser": "user",
	"remoteEnv": {"ZEPHYR_BASE":"/home/user/workspace/zephyrproject/zephyr","SHELL":"/bin/bash"},
	"workspaceMount": "source=${localWorkspaceFolder},target=/home/user/workspace/zephyrproject,type=bind,consistency=delegated",
	"workspaceFolder": "/home/user/workspace/zephyrproject/",
}

.vscode/c_cpp_properties.json
{
    "configurations": [
        {
            "name": "Linux",
            "includePath": [
                "${workspaceFolder}/**"
            ],
            "defines": [],
            "compilerPath": "/usr/bin/gcc",
            "cStandard": "c11",
            "cppStandard": "c++14",
            "intelliSenseMode": "gcc-x86",
            "compileCommands": "${workspaceFolder}/build/compile_commands.json"
        }
    ],
    "version": 4
}

.vscode/launch.json
{
    "version": "0.2.0",
    "configurations": [
        {
            "name": "Debug native_posix",
            "type": "cppdbg",
            "request": "launch",
            "program": "${workspaceFolder}/build/zephyr/zephyr.elf",
            "args": [],
            "stopAtEntry": false,
            "cwd": "${workspaceFolder}",
            "environment": [],
            "externalConsole": false,
            "MIMode": "gdb",
            "setupCommands": [
                {
                    "description": "Enable pretty-printing for gdb",
                    "text": "-enable-pretty-printing",
                    "ignoreFailures": true
                }
            ],
            "miDebuggerPath": "/usr/bin/gdb"
        }
    ]
}

After opening in VSCode with the plugin VSCode Remote Container, the dev environment is ready to use west init.
I did try to find a solution to make a configurable build task in vscode.
I found a litte hack with the settings.json to configure the task.json.
But to be honest, I still use the command line to build my projects.
.vscode/settings.json
{
	"BOARD":"native_posix",
	"PROJECT":"zephyr/samples/hello_world"
}

.vscode/tasks.json
{
	"version": "2.0.0",
	"tasks": [
		{
			"label": "format file (uncrustify)",
			"type": "shell",
			"linux": {
				"command": "uncrustify",
				"args": [
					"--replace",
					"--no-backup",
					"-l",
					"C",
					"-c",
					"${env:ZEPHYR_BASE}/.uncrustify.cfg",
					"${file}"
				]
			},
			"problemMatcher": []
		},
		{
			"label": "check code style file (checkpatch)",
			"type": "shell",
			"command": "${env:ZEPHYR_BASE}/checkpatch.pl",
			"args": [
				"--no-tree",
				"-f",
				"${file}"
			],
			"problemMatcher": [],
			"options": {
				"cwd": "${env:ZEPHYR_BASE}"
			}
		},
		{
			"label": "check code style file fix inplace (checkpatch)",
			"type": "shell",
			"command": "${env:ZEPHYR_BASE}/scripts/checkpatch.pl",
			"args": [
				"--no-tree",
				"-f",
				"${file}",
				"--fix-inplace"
			],
			"problemMatcher": [],
			"options": {
				"cwd": "${env:ZEPHYR_BASE}"
			}
		},
		{
			"label": "rebuild",
			"type": "shell",
			"group": "build",
			"linux": {
				"command": "west",
				"args": [
					"build",
					"-p",
					"auto",
					"-b",
					"${config:BOARD}",
					"${config:PROJECT}",
					"--",
					"-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
				]
			},
			"problemMatcher": {
				"source": "west-build",
				"base": "$gcc",
				"fileLocation": [
					"autodetect",
					"${workspaceFolder}/build"
				]
			}
		},
		{
			"label": "rebuild debug",
			"type": "shell",
			"group": "build",
			"linux": {
				"command": "west",
				"args": [
					"build",
					"-p",
					"auto",
					"-b",
					"${config:BOARD}",
					"${config:PROJECT}",
					"--",
					"-DOVERLAY_CONFIG=${workspaceFolder}/debug.conf",
					"-DCMAKE_EXPORT_COMPILE_COMMANDS=ON"
				]
			},
			"problemMatcher": {
				"source": "west-build",
				"base": "$gcc",
				"fileLocation": [
					"autodetect",
					"${workspaceFolder}/build"
				]
			}
		},
		{
			"label": "build",
			"type": "shell",
			"group": "build",
			"linux": {
				"command": "west",
				"args": [
					"build"
				]
			},
			"problemMatcher": {
				"source": "west-build",
				"base": "$gcc",
				"fileLocation": [
					"autodetect",
					"${workspaceFolder}/build"
				]
			}
		},
		{
			"label": "clean",
			"type": "shell",
			"group": "build",
			"linux": {
				"command": "rm",
				"args": [
					"-rf",
					"${workspaceFolder}/build"
				]
			},
			"problemMatcher": []
		},

	]
}

I use a customized docker image, that's why the officially zephyr build image will not work for some tasks/debug because of missing tools like standard installation for gdb or uncrustify.
I was also able to use west flash in the docker over pyocd.
See https://github.com/pyocd/pyOCD/blob/main/docs/remote_probe_access.md for more information.
On windows I started the debug server:
pyocd server --allow-remote
In the docker shell you can flash then with :
west flash --runner pyocd --tool-opt="-uremote:<host-ip>
Since https://github.com/Marus/cortex-debug/pull/435 is merged,
it should be possible (I used that config in the past with a hack) to use the debug plugin for cortex-m with the pyocd server:
.vscode/launch.json
        {
            "type": "cortex-debug",
            "request": "launch",
            "name": "pyocd nucleo_g071rb",
            "cwd": "${workspaceRoot}",
            "serverArgs": [
                "-O reset_type=hw",
                "-O connect_mode=under-reset",
                "-uremote:<host-ip>",
                "--target stm32g071rbtx"
            ],
            "executable": "${workspaceRoot}/build/zephyr/zephyr.elf",
            "servertype": "pyocd",
            "runToMain": true,
            "svdFile": "/home/user/cmsis-svd/data/STMicro/STM32G07x.svd"
        }

You need also to extend the settings.json to use the sdk compiler for arm.
.vscode/settings.json
{
	"cortex-debug.armToolchainPrefix": "arm-zephyr-eabi",
	"cortex-debug.armToolchainPath": "/opt/toolchains/zephyr-sdk-0.12.4/arm-zephyr-eabi/bin/",
}

But for me the best solution so far was to debug cortex-m targets with Segger Ozone.
It has a bit more embedded specific features compared with vscode and it is easier to setup.
Ozone has the possibility to redirect the path in the elf file to a windows path in your project configuration:
.ozone/debug.jdebug
Project.AddPathSubstitute ("//wsl$/Ubuntu-20.04/home/%USERNAME%/workspace/zephyrproject/.ozone", "$(ProjectDir)");`
Project.AddPathSubstitute ("/home/user","//wsl$/ubuntu-20.04/home/%USERNAME%");

With this configuration Ozone on windows works just fine with the in the container built elf.
Segger has a similar feature like pyocd server for their jlink, which I have not tested yet.
It is also possible to share the credentials for git into the docker container.
Further information for that:
https://code.visualstudio.com/docs/remote/containers#_sharing-git-credentials-with-your-container
What I also really like is the same docker image can be used for ci.

There's a difference in concept here:

Zephyr does not support "in-tree" builds means that you cannot build and generate the object files scattered around in the corresponding source directories, all the build files need to be under one common build/ (or whatever name you choose) folder
nRF Connect SDK using "out-of-tree" code means that it adds functionality to Zephyr in a separate Git repository by using west and its extension mechanism

Just to let you know that Nordic has released a preview of an extension bundle for vscode. It covers a number of features discussed in this thread.
More details in the blog post:
https://devzone.nordicsemi.com/nordic/nordic-blog/b/blog/posts/nrf-connect-for-visual-studio-code-preview
There is also a few videos that demonstrate the usage:
https://www.youtube.com/playlist?list=PLx_tBuQ_KSqEt7NK-H7Lu78lT2OijwIMl

@wbober This plug-in seems to be limited to nrf, not a general plug-in for zephyr? Is the plug-in implementation open source?

@hongshui3000 The plugin is not limited to nrf. Our SDK is based on Zephyr and you can work with any upstream sample or hardware platform without limitations. We have not made a decision to open-source it yet. Is there any particular functionality that you would like to change or expand?

Reading the post it sounds like this extension is leveraging Cortex-Debug for the debugging support (which many of us use!)

Correct, the extension is using cortex-debug. Our extension simply provides a context-aware configuration to cortex-debug, so you don't have to configure it manually.

The serial terminal seems to have no input function, I cannot send data to MCU.
It would be better if people can customize some trace functions by themselves. I hope to parse some log trace information according to my own rules.

Which terminal you refer to? The nRF Terminal extension that is bundled does support input for both serial and RTT. I've seen requests to add customization to log parsing before. What do you think about a list of regex rules that would filter out log input?

Serial terminal

Reading the post it sounds like this extension is leveraging Cortex-Debug for the debugging support (which many of us use!)

Correct, the extension is using cortex-debug. Our extension simply provides a context-aware configuration to cortex-debug, so you don't have to configure it manually.

The serial terminal seems to have no input function, I cannot send data to MCU.
It would be better if people can customize some trace functions by themselves. I hope to parse some log trace information according to my own rules.

Which terminal you refer to? The nRF Terminal extension that is bundled does support input for both serial and RTT. I've seen requests to add customization to log parsing before. What do you think about a list of regex rules that would filter out log output?

Serial terminal。It is best to customize the parsing script outside the plug-in, so that I can save the log file in the way I want. For example, I can use log to generate a test report, etc.
In addition, it is best to support serial shell terminal, interactive debugging is also very useful

Another issue seems to be that after installing the plug-in under Windows, running west under the command line under vscode does not work.I still suggest that there is a place to raise an issue, even if the project is not open source . :)

@hongshui3000 Yes, that was supposed to be a feature but turned out to be an issue that has hit users hard. We already have a workaround: if you use a manual toolchain installation we won't override the terminal environment. For a more complete solution we would like to introduce a separate terminal profile for NCS and leave the default one unmodified.
Feel free to open issues on https://devzone.nordicsemi.com/

https://www.rajeevpiyare.com/posts/nrfconnect-vs-code/

espressif has an open source implementation, but it does not seem to be based on zephyr
https://github.com/espressif/vscode-esp-idf-extension

Might be interesting for people who are using WSL2 with vscode:
https://devblogs.microsoft.com/commandline/connecting-usb-devices-to-wsl/

Would the Nordic folks be so kind as to open-source their VSCode extensions?

@ddavidebor Does the closed source nature of the extensions limits your usage in any way?

@wbober not for Nordic products, but of course the extensions could be expanded to encompass a larger number of toolchains over time - acting as the starting point for full VSCode integration
I totally understand if that's not viable for business reasons.

I wish there was a zephyr community maintained vscode extension

@ddavidebor There is no plan to open source the extension.
We are open to providing a limited support for development on other platforms. If you are able to provide more concrete examples of things that are missing I will take that feedback to the team. Perhaps there are ways to support with opening some APIs for extension with plugins or alike.
Can't make any promises but I'm open for discussion.

import vsc
https://github.com/smrtos/Zephyr2VSC

I still see this issue: https://github.com/microsoft/vscode-cpptools/issues/9187
E.g. whenever I use LOG_INF("Hello World") with only one argument, I get:

too few arguments in invocation of macro "Z_FOR_LOOP_1"

@mbrunnen you can get around this by adding __CDT_PARSER__ to your list of defines for CppTools. It's not a proper fix, but at least it makes the C++ extension usable.

@mbrunnen you can get around this by adding __CDT_PARSER__ to your list of defines for CppTools. It's not a proper fix, but at least it makes the C++ extension usable.

@trond-snekvik, I just tried that since I have the same issue, but I does not seem to have any effect, I still get the "too few arguments" warnings.
My configuration looks something like this:
{
    "name": "app",
    "cStandard": "c11",
    "cppStandard": "c++20",
    "intelliSenseMode": "linux-gcc-arm",
    "compilerPath": "${env:zephyrCompilerPath}",
    "compileCommands": "${workspaceFolder}/build/compile_commands.json",
    "defines": [
        "__CDT_PARSER__"
    ],
    "forcedInclude": [
        "${workspaceFolder}/build/autoconf.h"
    ]
},

@aunsbjerg according to the cpptools reference, compileCommands replaces all defines and includes, unfortunately. The extension only uses your custom defines and includes when it can't find the file you're editing in the compileCommands. I'm not sure the same applies to the forcedInclude though, so you might be able to achieve the same thing by creating a dummy header with this define and force include it.
Without compileCommands, it works for me on the current main branch, because of this section:
https://github.com/zephyrproject-rtos/zephyr/blob/61b5566e745c80c6bf6989ca23efcf032a369309/include/zephyr/logging/log.h#L439-L459
For what it's worth, clangd's configuration system is a lot more flexible, as it allows you to add or remove defines and includes in a yaml config file. The syntax is a little hard to work out, but their extension has come a long way, and often works better than Microsoft's in my experience.

@aunsbjerg according to the cpptools reference, compileCommands replaces all defines and includes, unfortunately. The extension only uses your custom defines and includes when it can't find the file you're editing in the compileCommands. I'm not sure the same applies to the forcedInclude though, so you might be able to achieve the same thing by creating a dummy header with this define and force include it.
Without compileCommands, it works for me on the current main branch, because of this section:
https://github.com/zephyrproject-rtos/zephyr/blob/61b5566e745c80c6bf6989ca23efcf032a369309/include/zephyr/logging/log.h#L439-L459
For what it's worth, clangd's configuration system is a lot more flexible, as it allows you to add or remove defines and includes in a yaml config file. The syntax is a little hard to work out, but their extension has come a long way, and often works better than Microsoft's in my experience.

Forced includes are also ignored when using compile_command.json. I played around with it, but couldn't really find a working solution. I'll give clangd a shot - thanks for the heads up and for taking the time to respond :)

Hey @trond-snekvik and @aunsbjerg, thanks for the help. __CDT_PARSER__ does help, but not using compile_commands.json is not really an option for me. I will try it with clangd :)

I started working on turning https://github.com/smrtos/Zephyr2VSC into a VSCode extension - "configurationProvider" - but oooooh boy things did not start smoothly!
Sanity check: I am getting good IntelliSense with "ms-vscode.cpptools" alone and simply defining compileCommands in .vscode/c_cpp_properties.json:
{
    "version": 4,
    "configurations": [
        {
            "name": "zephyr-default",
            "compileCommands": "${workspaceFolder}/build/compile_commands.json"
        }
    ]
}

This is nice, because I find cmake-tools cumbersome and the Nordic stuff completely unusable 😝.
On the other hand, I'm sure that I haven't used this config enough to discover why it's insufficient!  😭
Looking forward to trying out the new offerings from Microsoft when I have time: https://marketplace.visualstudio.com/items?itemName=ms-vscode.vscode-embedded-tools
Then again, maybe we should all just calm tf down before we turn VSCode into Eclipse...

FYI @robotdad, who works on the Embedded Tools extension!

@JPHutchins If you are interested in configurationProvider, you can also take a look at this extended method.Although it is not targeting zephyr

I still see this issue: microsoft/vscode-cpptools#9187
E.g. whenever I use LOG_INF("Hello World") with only one argument, I get:

too few arguments in invocation of macro "Z_FOR_LOOP_1"

interestingly (and sadly) this also fails when running CodeQL against a Zephyr application :(

Quite nice repository to get started with Zephyr debugging in vscode:
https://github.com/beriberikix/zephyr-vscode-example/tree/main

You can also try mine: https://github.com/KozhinovAlexander/zephyr_vscode_workspace
One thing to think about: default openocd v0.11.0 for ubuntu does not work with Marus/cortex.debug - you need to build v0.12.0 or install separately.

Note that there is a discussion regarding the same topic:
https://github.com/zephyrproject-rtos/zephyr/discussions/74005

authors: JPHutchins KozhinovAlexander SebastianBoe andrewboie aunsbjerg beriberikix carlescufi ddavidebor erwango func4plus1 hongshui3000 joerchan kartben manoj153 markusr marnold-b mbrunnen nashif rosterloh sigvartmh sslupsky trond-snekvik ulfalizer wbober whati001

repo: zephyrproject-rtos/zephyr


text:
json: json_obj_parse will modify the input string

Describe the bug
The end " of string items will be changed to \0 after json_obj_parse, i.e.  we have a char array like this:
{
    "method": "hello",
    "id": 1
}

if using json_obj_parse to proceed it, then it will be changed to
{
    "method": "hello�, //here " symbol changed to "\0"
    "id": 1
}

To Reproduce
Steps to reproduce the behavior:

create a json char array with string items
decode it with json_obj_parse
print out the json char array
see modifications

Expected behavior
From my point of view, json_obj_parse should be
int json_obj_parse(const char *payload, size_t len,
		   const struct json_obj_descr *descr, size_t descr_len,
		   void *val)

which means the input json string should not be modified.
Impact
Sometimes we'd like to proceed a json string twice:

the first time only proceed the first filed to know the info of this json
according the info we parse again with specific json_obj_descr
If json_obj_parse will modify string, user have to create a copy of string .

Environment (please complete the following information):

OS: Linux
Toolchain: Zephyr SDK
Commit SHA or Version used: f05cbb421ddfea03961b552977e4336d0ebdef63

@jhedberg I think the labels need a -bug +question and this issue can be closed.

(Not sure why I was assigned)

@dcpleung unfortunately I can't remember, but seems like a mistake - sorry about that.

@jhedberg I think the labels need a -bug +question and this issue can be closed.

@mbolivar agreed.

source: gharchive/issue

authors: codingspirit jhedberg mbolivar

license_type: permissive

license_source: github-api


text:
Cannot request update when writing to external flash

Describe the bug
Nordic's qspi does not support writing if data comes from internal flash. The buffer must be in RAM.
https://github.com/zephyrproject-rtos/hal_nordic/blob/master/nrfx/drivers/src/nrfx_qspi.c#L355
When the image magic is written the buffer resides in flash.
https://github.com/zephyrproject-rtos/zephyr/blob/master/subsys/dfu/boot/mcuboot.c#L100
https://github.com/zephyrproject-rtos/zephyr/blob/master/subsys/dfu/boot/mcuboot.c#L386
To Reproduce
Steps to reproduce the behavior:
Create any setup where dfu is done through the external flash.
Expected behavior
Image magic should be writable.
Impact
Showstopper
Screenshots or console output
N/A
Environment (please complete the following information):
master
Additional context
N/A

@nvlsianpu

It is enough to write to flash through the buffer in RAM. I copier the data to stack allocated array and performed write. Everything works. I think this is the best change to do so there will be no additional complexity done on the qspi driver side.
The other way to do it would be to handle it in the driver (detect that the buffer is not compatible and make a local copy in RAM). This may be more generic.

I thought that https://github.com/zephyrproject-rtos/zephyr/pull/25971/ introduced support for from flash writes.
@pabigot

AFAIK #25971 works, but if the amount of data being written is 4 or more bytes you need to set CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE=16` or whatever size buffer you want for the stack-based RAM region.  Only smaller values (which I know are used in MCUBoot) are handled without that option.

I see. CONFIG_NORDIC_QSPI_NOR_STACK_WRITE_BUFFER_SIZE is not yet present downstream so I have not seen it.

It's a forward-port of an equivalent functionality that I thought made it into 1.3, but maybe it didn't.  @hakonfam

There is NORDIC_QSPI_NOR_FLASH_ALLOW_STACK_USAGE_FOR_DATA_IN_FLASH in 1.3 indeed. I overlooked it.
When enabled magic is written correctly. Thanks @pabigot

authors: nvlsianpu pabigot pdunaj


text:
test: drivers: i2c: i2c_slave_api:

Describe the bug
We introduced new board to our test bench: nucleo_f091rc
And we get status failed on test: tests/drivers/i2c/i2c_slave_api/
Looking at original commit introducing this test:
(sha1: 94d3156540d27392236044eebcb4c3a6e6065185)

test: i2c: Add i2c_slave_api test

Add a new test to make I2C transfers between an I2C controller and
an I2C slave driver, either by:
- Using a virtual I2C driver
- Using a physical link and 2 physical controllers

The Physical test uses :
- Two EEPROM Slave drivers each connected to one I2C controller
- Both I2C controllers also act as master and take turns writing
to the opposite EEPROM Slave.
All of this connected physically on the board.

First issue : specifically for nucleo_f091rc and also for stm32f072b_disco, virtual I2C is disabled
CONFIG_I2C_VIRTUAL=n
Thus Physical EEPROM should be required, whereas no eeprom is connected to the board.
I could either:

exclude both board from this test, but the whitelist in yaml contains only those boards !! Thus test would become useless.
This is probably a 2nd issue, I don't understand why only those 2 boards are whitelisted.
Other board (not only stm32) could be tested either with physical eeprom or with virtual I2C.
Note: if external hardware is required maybe a 'harness' or 'fixture' should be added

use virtual I2C for those boards.
Unfortunately it doesn't work.
I don't think issue is specific to STM32, but simply not tested on other boards.
Possibly this test has not been run with virtual i2C for ages ...
@henrikbrixandersen it seem you use this test ?

To Reproduce
Steps to reproduce the behavior:

either remove CONFIG_I2C_VIRTUAL=n from file nucleo_f091rc.conf, or not
west build -p auto -b nucleo_f091rc tests/drivers/i2c/i2c_slave_api/
west flash
See error

Expected behavior
A clear and concise description of what you expected to happen.
Logs and console output
*** Booting Zephyr OS build zephyr-v2.3.0-980-gb08309e9924c  ***
Running test suite test_eeprom_slave
===================================================================
START - test_eeprom_slave
I: Found I2C Master device I2C_1
I: Found I2C Master device I2C_2
I: Found EEPROM device EEPROM_SLAVE_0
I: Found EEPROM device EEPROM_SLAVE_1
D: i2c: slave registered
I: EEPROM EEPROM_SLAVE_0 Attached !
D: i2c: slave registered
I: EEPROM EEPROM_SLAVE_1 Attached !
I: Start full read. Master: I2C_1, address: 0x56
D: stm32_i2c_msg_write: NACK

Environment (please complete the following information):

OS:  linux
Toolchain Zephyr SDK
Commit SHA: dc43c298a2

@ABOSTM Sorry, no I have never used this test not the I2C slave functionality. I may have fixed a few things in the test case when introducing I2C EEPROM support.

I don't understand why only those 2 boards are whitelisted.

I suspect because they were the ones originally used to develop the I2C follower interface in Zephyr.  The test pre-dates the testcast.yaml infrastructure and has never reflected the need to cross-connect I2C lines for two controllers to verify behavior (suggested in the original commit messages).
#26003 will support running this test on native posix with the existing simulator.  It may be best to simply remove the whitelist for the current test programs and replace them with native_posix.
Alternatively the test specification would have to be updated to note the need to wire two controllers together, then verify that it actually works.
There's evidence in the discussion of #26003 that the driver used to simulate follower behavior has bugs.  That should also be investigated.

authors: ABOSTM henrikbrixandersen pabigot


text:
tests-ci : libraries: libc: newlib_nano.thread_safety.userspace.stress test Timeout

Describe the bug
newlib_nano.thread_safety.userspace.stress test is Timeout on zephyr-v3.4.0-3926-g4ba97c22555e on mimxrt595_evk_cm33
testcase path is libraries/libc/newlib_nano/thread_safety/userspace/stress/malloc_thread_safety
see logs for details
To Reproduce
1.
scripts/twister --device-testing --device-serial /dev/ttyACM0 -p mimxrt595_evk_cm33  --sub-test libraries.libc

or
# cd tests/libraries/libc/newlib_nano/thread_safety/userspace/stress/malloc_thread_safety
# west build -b mimxrt595_evk_cm33
# west flash

See error

Expected behavior
test pass
Impact
Logs and console output
*** Booting Zephyr OS build zephyr-v3.4.0-3926-g4ba97c22555e ***
E: no free partition slots available
Running TESTSUITE newlib_thread_safety_locks
===================================================================
START - test_env_lock
PASS - test_env_lock in 0.001 seconds
===================================================================
START - test_malloc_lock
PASS - test_malloc_lock in 0.001 seconds
===================================================================
START - test_retargetable_lock_mutex
PASS - test_retargetable_lock_mutex in 0.001 seconds
===================================================================
START - test_retargetable_lock_sem
PASS - test_retargetable_lock_sem in 0.001 seconds
===================================================================
START - test_sfp_lock
PASS - test_sfp_lock in 0.001 seconds
===================================================================
START - test_sinit_lock
PASS - test_sinit_lock in 0.001 seconds
===================================================================
START - test_tz_lock
PASS - test_tz_lock in 0.001 seconds
===================================================================
TESTSUITE newlib_thread_safety_locks succeeded
Running TESTSUITE newlib_thread_safety_stress
===================================================================
START - test_malloc_thread_safety
Created 64 worker threads.
Waiting 30 seconds to see if any failures occur ...
E: ***** MPU FAULT *****
E:   Data Access Violation
E:   MMFAR Address: 0x301a0700
E: r0/a1:  0x301a0700  r1/a2:  0x301a0700  r2/a3:  0x0000000c
E: r3/a4:  0x301a0701 r12/ip:  0x1800f563 r14/lr:  0x1800ea19
E:  xpsr:  0x61000000
E: Faulting instruction address (r15/pc): 0x1800e504
E: >>> ZEPHYR FATAL ERROR 0: CPU exception on CPU 0
E: Current thread: 0x30180fd8 (unknown)
E: Halting system

Environment (please complete the following information):

OS: (e.g. Linux )
Toolchain (e.g Zephyr SDK)
Commit SHA or Version used: zephyr-v3.4.0-3926-g4ba97c22555e

does not reproduce on latest main 61f065230ad9bca91633cd57b9dcd5e90c5d8bc7

authors: decsny hakehuang


text:
need to improve filtering and coverage in sanitycheck

The numbers of code we build and run per commit has been growing out of control and the promise of having CI results delivered in 15 minutes or less is not being met anymore (we are close to 30 minutes now).
We need a better way to filter tests and still get good coverage.
See the how the numbers have been growing over time:

was addressed with --integration feature


text:
LE Audio: bap list should also show the states of each stream/endpoint

Is your enhancement proposal related to a problem? Please describe.
The bap list command is a nice way to get an overview of the existing streams and endpoints, but it does not show the states of them, which makes it hard to tell what commands can be done on which streams.
Describe the solution you'd like
The command should use the bt_bap_ep_get_info to get the state of each stream/endpoint and show that as well.
Describe alternatives you've considered
N/A
Additional context
N/A

I will take a look at this issue

authors: Thalley jthm-ot


text:
arc: arm: fix support for MPUs on non-XIP systems

We now use MPU_ALIGN() to set the location counter after the ROM region.
On non-power of two systems, this just rounds the location up to the MPU granularity, the parameter value is ignored.
On power of two systems, this rounds up appropriately based on the size of ROM.
Fixes: #15558
Signed-off-by: Andrew Boie andrew.p.boie@intel.com

I found a much simpler way to do this entirely in the linker script.
I'll take off 'draft' status once I've done some more testing.

recheck

source: gharchive/pull-request


text:
Fix coverity issue in coap_client sample

Ignore socket close() return value in this sample.
Fixes #18961

We might want to backport this (and other stuff) to 2.0.1, if we are to have that.

We might want to backport this (and other stuff) to 2.0.1, if we are to have that.

Yep, that's why I did not merge it yet. We need to have the backport bot running for 2.0.1

As agreed in release meeting, only really major issues would go to 2.0.1 and this is not that stuff.

authors: ioannisg jukkar rveerama1


text:
arm: sample apps and build support for PSA TF-M and Zephyr

This PR enables CI checks to pass for TF-M, and introduces build support to automatically pull in the external TF-M repository and all dependencies, and perform a build of TF-M as part of a standard Zephyr sample app build procedure. It is based on TF-M 1.0-rc2.

This should be a west project and a zephyr module. Please open an issue to create a TF-M repo under the zephyrproject-rtos and then be able to point west.yml to it. Please follow the instructions here:
https://docs.zephyrproject.org/latest/guides/modules.html

Hi,
The TF-M project and it's related repos are standalone projects. And the TF-M is not a module or a part of Zephyr. TF-M run in secure execution and Zephyr run in nonsecure execution, the compile seperately to different binaries. May be this requires more discussions or ideas on this point for further. Anyway, created an issue for new module request.
Thanks

@microbuilder I am working on Musca B1 to make it reproduce psa arch tests pass. The Zephyr is 2.0.0. Output like below:
`[Sec Thread] Secure image initializing!
***** Booting Zephyr OS build zephyr-v2.0.0-1393-gec3f603b8a7a *****
TF-M PSA Arch Tests with Zephyr on musca_b1_nonsecure
***** PSA Architecture Test Suite - Version 0.9 *****
Running.. Crypto Suite

....
************ Crypto Suite Report **********
TOTAL TESTS     : 44
TOTAL PASSED    : 43
TOTAL SIM ERROR : 0
TOTAL FAILED    : 0
TOTAL SKIPPED   : 1

Entering standby..
`

What is this commit for ?
"Merge github.com:zephyrproject-rtos/zephyr into psa_arch_tests_for_tf…"

This is a consequence of updating the PR against master, pulling recent changes into the PR ... I can remove this and most other superfluous commit messages before the final PR review to clean this up.

This should be a west project and a zephyr module. Please open an issue to create a TF-M repo under the zephyrproject-rtos and then be able to point west.yml to it. Please follow the instructions here:
https://docs.zephyrproject.org/latest/guides/modules.html

Hi Carles ... I understand the idea behind the module request, but I would argue in this case that this is specifically a sample with some BSP fixes thrown on top, and not a module as such since it doesn't add TFM in the PR itself. The cmake file does do some work to pull TF-M in as an external project as part of the normal Zephyr build process, build it with appropriate options, and link against that for the secure processing environment, but the core code here is exclusively a sample application for Zephyr, which doesn't seem inappropriate to me to include in the samples folder(?).

Why combine all commit history into one in the PR?

Why combine all commit history into one in the PR?

It makes the commit easier to refer to in the future once it gets merged in, versus 15 related but overlapping commits.

Why combine all commit history into one in the PR?

It makes the commit easier to refer to in the future once it gets merged in, versus 15 related but overlapping commits.

Hi Kevin,
I do not think so. The commits means history and the steps of how the feature enabled. All patches are trackable by git log. And also easier for reviewer. I do combine some patches because they just fix few words that in the same document. For other patches, I'd prefer keep them as original.

So I think we should see about having TF-M as a west module and setting up a repo under the zephyr project that tracks the TF-M repo (and maybe combines all the components that TF-M needs as its own west manifest).

Hi,
If put TFM and all dependencies as modules, is that means everyone will download them by west update, is it? And the modules seem using the zephyr CMake style to build each module, I think to do the same for TF-M project is a big effort. The TFM eventually not as a part of zephyr.bin, How do you think about the way we using now? To keep TF-M independent pull, and compile.

If put TFM and all dependencies as modules, is that means everyone will download them by west update, is it? And the modules seem using the zephyr CMake style to build each module, I think to do the same for TF-M project is a big effort. The TFM eventually not as a part of zephyr.bin, and all dependencies projects size is large, so implement all the work like it shows in this PR. To keep TF-M independent pull, and compile.

Yes, the modules would be installed via the usual west update command, and most likely this will require a fork of trusted-firmware-m.git into a new project like zephyrproject-rtos/tfm to host the TF-M code, and any support files similar to what happens with open-amp today: https://github.com/zephyrproject-rtos/open-amp

If put TFM and all dependencies as modules, is that means everyone will download them by west update, is it? And the modules seem using the zephyr CMake style to build each module, I think to do the same for TF-M project is a big effort. The TFM eventually not as a part of zephyr.bin, and all dependencies projects size is large, so implement all the work like it shows in this PR. To keep TF-M independent pull, and compile.

Yes, the modules would be installed via the usual west update command, and most likely this will require a fork of trusted-firmware-m.git into a new project like zephyrproject-rtos/tfm to host the TF-M code, and any support files similar to what happens with open-amp today: https://github.com/zephyrproject-rtos/open-amp
This would have the advantage of making it clearer which commit of TF-M we are currently using, and we can also add the TF-M dependencies into this new repo (mbed-crypto, CMSIS_5, etc.) so that we no longer rely on an awkward git clone solution to build.

I know that, and was wrok on the openamp module. Since that script can be updated. https://github.com/zephyrproject-rtos/zephyr/commit/469ad5218be949ee421dc6d6355d5553fbd7ee5b

So I think we should see about having TF-M as a west module and setting up a repo under the zephyr project that tracks the TF-M repo (and maybe combines all the components that TF-M needs as its own west manifest).

Hi,
If put TFM and all dependencies as modules, is that means everyone will download them by west update, is it? And the modules seem using the zephyr CMake style to build each module, I think to do the same for TF-M project is a big effort. The TFM eventually not as a part of zephyr.bin, and all dependencies projects size is large, so implement all the work like it shows in this PR. To keep TF-M independent pull, and compile.

If put it in module, west update will have 730 MB larger for every one to include TF-M and it's dependent projects.
All Cmake files in TFM does not using zephyr_xxxx() now, and I do not think it is vaulable to rework all Cmake files on a specific TF-M version, TF-M evolves all the time.
tfm_s.bin is not a part in zephyr.bin, not sure if it is properly as a module.
Just my oponion on it, thanks.

So I think we should see about having TF-M as a west module and setting up a repo under the zephyr project that tracks the TF-M repo (and maybe combines all the components that TF-M needs as its own west manifest).

Hi,
If put TFM and all dependencies as modules, is that means everyone will download them by west update, is it? And the modules seem using the zephyr CMake style to build each module, I think to do the same for TF-M project is a big effort. The TFM eventually not as a part of zephyr.bin, and all dependencies projects size is large, so implement all the work like it shows in this PR. To keep TF-M independent pull, and compile.

If put it in module, west update will have 730 MB larger for every one to include TF-M and it's dependent projects.

we should be able to construct the tf-m module repo in such a way that its not that big, we don't need all of CMSIS for example.

All Cmake files in TFM does not using zephyr_xxxx() now, and I do not think it is vaulable to rework all Cmake files on a specific TF-M version, TF-M evolves all the time.
tfm_s.bin is not a part in zephyr.bin, not sure if it is properly as a module.
Just my oponion on it, thanks.

So I think we should look at a middle ground here to start with.  We should use west as the means of getting the TFM source, how we build it after that we can figure out.  I agree it doesn't make sense to introduce/change the TFM CMake files.  However we should be able to do something similar to openamp.

@tejlmand could you please take a look at the integration between the 2 CMake build systems?

@tejlmand could you please take a look at the integration between the 2 CMake build systems?

I havent been able to get ninja to do the TF-M build first, the way that make does. I believe the dependencies are being setup correctly, with the add_dependencies function:
ExternalProject_Add(
  tfm
  SOURCE_DIR ${TFM_BASE_DIR}
  CMAKE_ARGS -DPROJ_CONFIG=${TFM_BASE_DIR}/configs/ConfigRegressionIPC.cmake
             -DTARGET_PLATFORM=${TFMBOARD}
	     -DBL2=True
	     -DCOMPILER=GNUARM
	     #-DGNUARM_PREFIX=arm-zephyr-eabi
	     #-DGNUARM_PATH=${ZEPHYR_SDK_INSTALL_DIR}/arm-zephyr-eabi
	     -DGNUARM_PREFIX=arm-none-eabi
	     -DGNUARM_PATH=/usr/local/prog/gcc-arm-none-eabi-7-2018-q2-update
  BUILD_BYPRODUCTS ${BINARY_DIR}/tfm/veneers/s_veneers.o
)
add_dependencies(app tfm)

With ninja:
$ cd samples/tfm_integration/tfm_ipc/
$ mkdir build && cd build
$ cmake -GNinja -DBOARD=mps2_an521_nonsecure ..
$ ninja
ninja: error: 'tfm-prefix/src/tfm-build/install/export/tfm/veneers/s_veneers.o', needed by 'zephyr/zephyr_prebuilt.elf', missing and no known rule to make it

With make:
$ cmake -DBOARD=mps2_an521_nonsecure ..
$ make run
...
[TF-M build output]
...
[Sec Thread] Secure image initializing!
TF-M isolation level is: 1
Booting TFM v1.0
*** Booting Zephyr OS build v1.12.0-rc1-19023-g10461e3708e7  ***
The version of the PSA Framework API is 256.
The minor version is 1.
Connect success!
TFM service support minor version is 1.

@microbuilder I changed a little in the CMakeLists.txt to get it running locally with ninja
You can try to fetch: https://github.com/karl-zh/zephyr/pull/1 and do west update.
This only corrects the dependency issues.
Note, there are still some build issues that @karl-zh  should look into.

Made a couple of small suggestions for #19985 just to get it to build.
Those are found here:
karl-zh#1
zephyrproject-rtos/trusted-firmware-m#3

Thanks, tested on the musca_b1 and an521 build targets, and resolved the linker issue (it was due to the output path changing for the TF-M build), and works as expected now, once zephyrproject-rtos/trusted-firmware-m#3 is merged in.

Very good to see this work started.
I have some thoughts about the IPC sample. The main file is largely copied from test/suites/ipc/non_secure/ipc_ns_interface_testsuite.c in TF-M. I would really prefer that this file is used as is, from TF-M. This involves adding some more files to the build to allow calling the PSA functions by their proper name (not calling the *_veneer() functions directly). Also, AFAIU "IPC" in TF-M means calling the Secure APIs through a mailbox, which will need additional files added (see app/CMakeLists.txt for more info).
The most important thing IMO is allowing the Zephyr user to call PSA APIs as they would if they developed on TF-M directly, even if we don't support both IPC and library calls.
See this if you haven't before: https://ci.trustedfirmware.org/job/tf-m-build-test-nightly/lastSuccessfulBuild/artifact/build-docs/tf-m_documents/install/doc/user_guide/html/docs/user_guides/tfm_build_instruction.html#configurations

I agree this could be done differently, and I have two other demos that I need to get in, one using library mode, and the other to run the PSA FF API, etc., tests as a sample app, but this might need to happen after the 2.3 merge window closes Friday.

@oyvindronningstad I agree this could be done differently, and I have two other demos that I need to get in, one using library mode, and the other to run the PSA FF API, etc., tests as a sample app, but this might need to happen after the 2.3 merge window closes Friday.

Sounds good. The one thing I want to get in before the merge window is allowing the PSA APIs to be used by their proper names, so that once the PSA APIs are added, it doesn't break things for the user.

@microbuilder Placed some additional code, just to provide a starting point, without changing current design.
https://github.com/karl-zh/zephyr/pull/2
https://github.com/zephyrproject-rtos/trusted-firmware-m/pull/6
Basically, it move the ExternalProject_Add() to the TFM Zephyr module, and then calls trusted_firmware_build instead.
Now, the question, which I think @oyvindronningstad is also addressing, is how do we want to integrate TF-M into Zephyr in general.
Cause I don't assume we want to add trusted_firmware_build throughout Zephyr samples.
Also, there could be downstream users with a different solution, or not wanting to clone TF-M, and such a function will break the build, if it is missing.
Do you have some thoughts regarding the future integration that  you can share ?

which actually makes me notice there is no documentation accompanying this PR.
It should be documented how users can enable and use TF-M.

which actually makes me notice there is no documentation accompanying this PR.
It should be documented how users can enable and use TF-M.

There is some documentation at the board level, but no high-level TF-M documentation, I agree. I'll add something to get started in with the next push, once I'm done some further cleanup with the build system.

@microbuilder this needs to be moved out of Draft ASAP if it is to be merged in time for 2.3.

@microbuilder this needs to be moved out of Draft ASAP if it is to be merged in time for 2.3.

@carlescufi Making some final changes to it tonight, and hope to flip the switch tomorrow morning for review. Sorry for the delay.

@oyvindronningstad I agree this could be done differently, and I have two other demos that I need to get in, one using library mode, and the other to run the PSA FF API, etc., tests as a sample app, but this might need to happen after the 2.3 merge window closes Friday.

Sounds good. The one thing I want to get in before the merge window is allowing the PSA APIs to be used by their proper names, so that once the PSA APIs are added, it doesn't break things for the user.

@oyvindronningstad This commit should resolve the issue you brought up, and PSA API functions can now be called directly from the sample app: https://github.com/zephyrproject-rtos/zephyr/pull/19985/commits/ce2726d1b603157b4d1c62c61ec019613ce11424

@oyvindronningstad I also added a more complex 'real world' example making use of the attestation, crypto and secure storage modules here, accessing the PSA APIs with the normal names in IPC mode of course: https://github.com/zephyrproject-rtos/zephyr/pull/19985/commits/4e2173e3269f70559ae6e9ea79bbd3ebe5aaa582

Just a gentle reminder @microbuilder (in case you 've forgotten) there are some commit messages to be filled in.

Just a gentle reminder @microbuilder (in case you 've forgotten) there are some commit messages to be filled in.

@ioannisg Was just fixing those, done now.

Tagging as DNM because of the missing SHA in west.yml. Otherwise this looks good to go to me.

@ioannisg take another look please, commits have been rebased into a logical set

@microbuilder thanks for taking over this PR and for bringing this to merge-able state
@tejlmand @oyvindronningstad thanks for the reviews!

authors: carlescufi galak ioannisg karl-zh microbuilder oyvindronningstad tejlmand


text:
dma: stm32: implement get_status api

get_status api is not implemented in stm32 dma driver
but it will be used by others like async uart driver.
The changes need to be verified soon.

@erwango Please help to review.

Verified the change is working.


text:
net: openthread: Init NCP after USB communication is established

The device has sent RESET_POWER_UP message before the communication with the host has been established. It could be observed with pyspinel (with debugging enabled) which displayed Framing error.
This commit fixes the bug by initializing NCP after the host stated is ready to communicate.
For consistency the behavior is also changed for pure serial communication.

CI is not green. The output message is not clear if it is my fault an I should do sth about it or nor. Anyone help?

@carlescufi Could you help with the CI problem?

I rebased the PR few times and always buildkite/zephyr fail, but it seems the problem is in the CI scripts not the PR itself. Can anyone help?

I rebased the PR few times and always buildkite/zephyr fail, but it seems the problem is in the CI scripts not the PR itself. Can anyone help?

The source of CI problem was that the changed code was not covered by any test.