Skip to content

Commit

Permalink
add posix shell
Browse files Browse the repository at this point in the history
squashed & rebased version, not including:
- listener changes
- src/firmware renaming

Commits:

tag_to_version.py: fix Python3 error

subprocess.communicate returns bytes instead of a str which is not the
same for Python3. Therefore, we need to decode the bytes.

cmake: remove folder src/firmware

The folder src/firmware was not intuitive. Why would the binaries for
SITL be inside a src and why even inside a src/firmware folder. Also,
the rootfs was put there which made it even more confusing.

The CMakeLists.txt files are moved into cmake/ and get now called from
the main CMakeLists.txt.

qshell: support for return value

Instead of just sending commands, qshell will now also wait until
the command has finished on QURT and sent back a return value. This will
allow all modules on the DSP side to be spawned from the Linux side
meaning that we only need one config/startup file instead of two.

adb_upload: create folders before pushing

Previously the script failed if the folder on the destination was not
already existing. This therefore makes pushing easier.

posix: spawn PX4 modules in bash

This adds the possibility to spawn PX4 modules out of bash. Basically,
the main executable can now be started as a server/daemon or as a
client.
The server replaces the existing functionality of the main exe with
the pxh shell, however, it also opens a pipe that clients can talk to.

Clients can run or spawn PX4 modules or commands by connecting to the
server over the pipe. They clients will get the stdout and return value
of their commands via a client specific pipe back.

This work will allow to start all modules using a bash script similar to
the way it is done in NuttX where the NuttShell scripts the startup
scripts and starts the modules.

SITL: use new client shell in SITL

This is a first step to use the new shell capabilities for SITL.
The new startup bash script rcS merges (and therefore replaces) the two
existing scripts rcS_gazebo_iris and rcS_jmavsim_iris.

More cleanup will be necessary for the rest of the SITL startup scripts.

Snapdragon: use new shell to start all modules

Instead of different mainapp.config and px4.config files, we can now use
a unified rcS bash script which starts all the modules based on
parameters, mainly the SYS_AUTOSTART param.

Snapdragon: fix the airframe description

pxh: argv needs to end with a nullptr

The comment was wrong that argv needs an additional 0 termination.
Instead it needs a nullptr at the end.

px4_posix_tasks: variable cleanup

The px4_task_spawn_cmd function got a cleanup while debugging, however,
no functional changes.

Snapdragon: move some drivers to 4100 config

These drivers are supported by the community, so they go into the 4100
config.

Snapdragon: update 210qc platform

px4_daemon: use doxygen comments

apps.h_in: fix string printf: use .c_str()

px4_daemon: \b -> \n in printf

px4_daemon: handle error in generate_uuid (close the file on error)

posix main: some clarifications in comment (it's the symlinks not the script aliases)

cmake: remove new install command again

This one was probably wrong and untested. Installing needs revisiting.

POSIX: remove argument USES_TERMINAL

POSIX: copy init and mixer files for SITL

Instead of using non-working install commands, the mixer and startup
files are now copied as part of the build in cmake.

adb_upload.sh: remove leftover commented printf

POSIX main: just the pointer instead of memmove

POSIX main: remove chroot

chroot is removed because it hasn't been used anywhere and seems
untested.

px4_daemon: remove client pipe when cleaning up

px4_daemon: fail if the client pipe already exists

The client pipe is supposed to be specific (by UUID), so the path
shouldn't exist already.

history: limit the number of history entries

This is a protection to avoid filling the memory if we are entering a
lot of commands (e.g. auto-generated).

px4_daemon: add a threadsafe map and use it

px4_daemon: whitespace

px4_daemon: fix client parsing

Sometimes the client ends up reading more than one packet in one read.
The parsing is not made for this and would require a (ring)buffer for
it.

The solution of this commit just reads as much as needed from the pipe
which avoids having to do buffering and parsing.

posix: changes sitl_run.sh and main.cpp cleanup

This changes the paths in sitl_run.sh quite a bit to allow the px4
binary to run in the rootfs directory which should make it convenient
and very close to the NuttX variant.

Also main.cpp got a big cleanup after the big rebase with some
conflicts. Quite some functionality was removed but it has yet to be
seen if it needs to be re-added.

px4_log: cleanup log levels, now they make sense

Before DEBUG and INFO log levels where inverted which didn't make much
sense in my eyes.

dataman: fix path for bash shell

logger: fix paths for bash shell

mavlink: fix paths for bash shell

param: fix path for bash shell

inav: fix paths for bash shell

sdlog2: fix paths for bash shell

ROMFS: add forgotten mixer to list

SITL init: more models, more options

- Support for different models using the unified startup
script rcS.
- Support to choose the estimator by setting the environment variable
  PX4_ESTIMATOR.
- Support to choose the logger by setting the environment variable
  PX4_LOGGER.

rcS: fix string comparison

listener: use template file

Instead of having all of the C++ code inside the Python file it is
nicer to have a separate template file with the C++ headers, etc.

px4_log: add PX4_INFO_RAW for raw printfs

This allows to do custom formatting but is still transported over
sockets to clients.

topic_listener: use PX4_INFO_RAW instead of printf

commander: use PX4_INFO_RAW for status

listener: rewrite to classes and factory

posix: fix some argument warnings

generate_listener.py: by accident changed shebang

listener: big refactor of the generator

Hopefully this makes it easier to read and change in the future.

rcS: manually take over rebase changes

listener: remove leftover try

listener: properly clean up topic instance

rcS: take over some vehicle specific changes

posix-configs: vehicle specifics to separate files

posix-configs: remove leftover lines

uORBDevices: new PX4_INFO_RAW instead of printf

px4_log: just use printf on NuttX

listener: use less binary space, strip on NuttX

generate_listener.py: remove commented code

cmake: fix syntax error from merge

px4_daemon: fixes after rebase of apps.h/cpp fix

px4_daemon: namespace missing

posix: only create stub for fsync on QURT

unitests: reduce dependencies of param test

This makes the unit test compile and link again after the bash changes.

QURT: some compile fixes after a rebase

SITL: arg change for sitl_run.sh to use rcS_test

This allows to use a custom startup file for testing.

SITL: add the folder test_data

SITL: implement shutdown command as systemcmd

The shutdown command needs to be a proper systemcmd, otherwise the alias
and symlink generation doesn't work and we end up calling shutdown of
the host computer which is to be avoided.

px4fmu_test: same IO_pass mixer as px4fmu_default

px4fmu_test: use normal quad x mixer

There is no good reason to use a specific test mixer, except more cmake
code around it. Therefore just use the same mixer as default, and at
some point px4fmu_test and px4fmu_default can get merged

POSIX: cleanup, dir and symlink fixes

This cleans up the logic behind the symlinking and creating directories.

POSIX: correct arg order in usage info

tests: fix paths for SITL tests

POSIX: printf fix

sitl_run.sh: try to make this run on Mac as well

cmake: try to make jenkins happier

Path cleanup, the bin is no longer in src/firmware

POSIX: fix symlink logic

SITL: prefix all exported env variables

cmake: fix path for ROS tests

integrationtests: fix log path

launch: try to make tets with ROS working again

px4_defines: fix after wrong merge deconflicting

px4_defines: get paths for POSIX correct

cmake: fix cmake arguments

This was fine with cmake 3.6 but did not work with cmake 3.2.2

cmake: use cp instead of cmake -E copy

cmake -E copy does not support copying multiple files with versions <
3.5. Therefore, just use cp for now.

ROMFS: fix build error after rebase

cmake: fix paths in configs

launch: use `spawn_model` again

cmake: various fixes after big rebase

param: path fixes after rebase

posix platform: fixes after rebase

test_mixer: fix screwed up rebase
  • Loading branch information
julianoes authored and LorenzMeier committed Aug 8, 2018
1 parent 7bdfac7 commit 0c5c741
Show file tree
Hide file tree
Showing 62 changed files with 3,515 additions and 721 deletions.
14 changes: 14 additions & 0 deletions ROMFS/px4fmu_test/mixers/IO_pass.mix
Original file line number Diff line number Diff line change
@@ -1,24 +1,38 @@
Passthrough mixer for PX4IO
============================

This file defines passthrough mixers suitable for testing.

Channel group 0, channels 0-7 are passed directly through to the outputs.

M: 1
O: 10000 10000 0 -10000 10000
S: 0 0 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 1 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 2 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 3 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 4 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 5 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 6 10000 10000 0 -10000 10000

M: 1
O: 10000 10000 0 -10000 10000
S: 0 7 10000 10000 0 -10000 10000
25 changes: 0 additions & 25 deletions ROMFS/px4fmu_test/mixers/quad_test.mix

This file was deleted.

7 changes: 7 additions & 0 deletions ROMFS/px4fmu_test/mixers/quad_x.main.mix
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
R: 4x 10000 10000 10000 0
M: 1
O: 10000 10000 0 -10000 10000
S: 3 5 10000 10000 0 -10000 10000
M: 1
O: 10000 10000 0 -10000 10000
S: 3 6 10000 10000 0 -10000 10000
9 changes: 6 additions & 3 deletions Tools/adb_upload.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,13 +5,16 @@ if [[ "$#" < 2 ]]; then
exit
fi

# Get last argument
for last; do true; done

echo "Wait for device..."
adb wait-for-device
echo "Uploading..."

# Get last argument
for last; do true; done
echo "Creating folder structure..."
adb shell mkdir -p $last

echo "Uploading..."
# Go through source files and push them one by one.
i=0
for arg
Expand Down
4 changes: 2 additions & 2 deletions Tools/adb_upload_to_bebop.sh
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
#!/bin/bash

if [ -z ${BEBOP_IP+x} ]; then
if [ -z ${BEBOP_IP+x} ]; then
ip=192.168.42.1
echo "\$BEBOP_IP is not set (use default: $ip)"
else
else
ip=$BEBOP_IP
echo "\$BEBOP_IP is set to $ip"
fi
Expand Down
4 changes: 2 additions & 2 deletions Tools/decode_backtrace.py
Original file line number Diff line number Diff line change
Expand Up @@ -63,7 +63,7 @@ def usage():
msg = """
Usage: Tools/decode_backtrace.py <builddir>
This will load the symbols for <builddir>/src/firmware/posix/px4
This will load the symbols for <builddir>/bin/px4
The user just needs to copy and paste the backtrace into the terminal
where decode_backtrace.py is running.
Expand All @@ -75,7 +75,7 @@ def usage():

# Load the symbols from the binary
def load_symbol_map():
output = subprocess.check_output(["nm", "-p", "-C", os.sys.argv[1]+"/src/firmware/posix/px4"])
output = subprocess.check_output(["nm", "-p", "-C", os.sys.argv[1]+"/bin/px4"])
data = output.split("\n")
data.sort()

Expand Down
2 changes: 1 addition & 1 deletion Tools/sitl_multiple_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ while [ $n -le $sitl_num ]; do

pushd "$working_dir" &>/dev/null
echo "starting instance $n in $(pwd)"
sudo -b -u $user ../src/firmware/posix/px4 -d "$src_path" rcS >out.log 2>err.log
sudo -b -u $user ../bin/px4 -d "$src_path" rcS >out.log 2>err.log
popd &>/dev/null

n=$(($n + 1))
Expand Down
20 changes: 15 additions & 5 deletions Tools/sitl_run.sh
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@ set -e
echo args: $@

sitl_bin=$1
rcS_dir=$2
rcS_path=$2
debugger=$3
program=$4
model=$5
Expand All @@ -15,7 +15,7 @@ build_path=$7
echo SITL ARGS

echo sitl_bin: $sitl_bin
echo rcS_dir: $rcS_dir
echo rcS_path: $rcS_path
echo debugger: $debugger
echo program: $program
echo model: $model
Expand Down Expand Up @@ -63,7 +63,7 @@ fi

if [ "$#" -lt 7 ]
then
echo usage: sitl_run.sh rc_script rcS_dir debugger program model src_path build_path
echo usage: sitl_run.sh sitl_bin rcS_path debugger program model src_path build_path
echo ""
exit 1
fi
Expand Down Expand Up @@ -123,10 +123,18 @@ cd $working_dir
# Do not exit on failure now from here on because we want the complete cleanup
set +e

sitl_command="$sudo_enabled $sitl_bin $no_pxh $chroot_enabled $src_path $src_path/${rcS_dir}/${model}"
sitl_command="$sitl_bin $no_pxh $rootfs $rootfs/${rcS_path}"

echo SITL COMMAND: $sitl_command

# Prepend to path to prioritize PX4 commands over potentially already
# installed PX4 commands.
export PATH="$build_path/bin":$PATH

export PX4_SIM_MODEL=${model}

pushd $rootfs

if [[ -n "$DONT_RUN" ]]
then
echo "Not running simulation (\$DONT_RUN is set)."
Expand Down Expand Up @@ -156,9 +164,11 @@ then
echo "######################################################################"
read
else
$sitl_command
eval $sitl_command
fi

popd

if [[ -z "$DONT_RUN" ]]
then
if [ "$program" == "jmavsim" ]
Expand Down
1 change: 1 addition & 0 deletions cmake/configs/posix_rpi_common.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ set(config_module_list
systemcmds/ver
systemcmds/esc_calib
systemcmds/reboot
systemcmds/shutdown
systemcmds/topic_listener
systemcmds/tune_control
systemcmds/perf
Expand Down
1 change: 1 addition & 0 deletions cmake/configs/posix_sdflight_default.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -41,6 +41,7 @@ set(config_module_list
systemcmds/led_control
systemcmds/mixer
systemcmds/ver
systemcmds/shutdown
systemcmds/topic_listener
systemcmds/tune_control

Expand Down
4 changes: 3 additions & 1 deletion cmake/configs/posix_sitl_default.cmake
Original file line number Diff line number Diff line change
Expand Up @@ -34,6 +34,7 @@ set(config_module_list
systemcmds/perf
systemcmds/pwm
systemcmds/reboot
systemcmds/shutdown
systemcmds/sd_bench
systemcmds/top
systemcmds/topic_listener
Expand Down Expand Up @@ -147,7 +148,7 @@ set(config_module_list
# Default config_sitl_rcS_dir (posix_sitl_default), this is overwritten later
# for the config posix_sitl_efk2 and set again, explicitly, for posix_sitl_lpe,
# which are based on posix_sitl_default.
set(config_sitl_rcS_dir posix-configs/SITL/init/ekf2 CACHE INTERNAL "init script dir for sitl")
set(config_sitl_rcS_dir etc/init/rcS CACHE INTERNAL "init script dir for sitl")

set(config_sitl_viewer jmavsim CACHE STRING "viewer for sitl")
set_property(CACHE config_sitl_viewer PROPERTY STRINGS "jmavsim;none")
Expand All @@ -162,3 +163,4 @@ if(REPLAY_FILE)
message("Building with uorb publisher rules support")
add_definitions(-DORB_USE_PUBLISHER_RULES)
endif()

4 changes: 0 additions & 4 deletions cmake/configs/posix_sitl_ekf2.cmake
Original file line number Diff line number Diff line change
@@ -1,5 +1 @@
include(cmake/configs/posix_sitl_default.cmake)

set(config_sitl_rcS_dir
posix-configs/SITL/init/ekf2
)
80 changes: 80 additions & 0 deletions cmake/posix/apps.h_in
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
/* builtin command list - automatically generated, do not edit */
#include <string>
#include <map>
#include <stdio.h>

#include <px4_tasks.h>
#include <px4_posix.h>
#include <px4_log.h>
#include <stdlib.h>

using namespace std;

extern void px4_show_devices(void);

extern "C" {
${builtin_apps_decl_string}
static int list_tasks_main(int argc, char *argv[]);
static int list_files_main(int argc, char *argv[]);
static int list_devices_main(int argc, char *argv[]);
static int list_topics_main(int argc, char *argv[]);
static int sleep_main(int argc, char *argv[]);
}

static map<string,px4_main_t> app_map(void)
{
static map<string,px4_main_t> apps;

${builtin_apps_string}
apps["list_tasks"] = list_tasks_main;
apps["list_files"] = list_files_main;
apps["list_devices"] = list_devices_main;
apps["list_topics"] = list_topics_main;
apps["sleep"] = sleep_main;

return apps;
}

map<string,px4_main_t> apps = app_map();

static void list_builtins(void)
{
printf("Builtin Commands:\n");
for (map<string,px4_main_t>::iterator it=apps.begin(); it!=apps.end(); ++it)
printf("\t%s", it->first.c_str());
}

static int list_tasks_main(int argc, char *argv[])
{
px4_show_tasks();
return 0;
}

static int list_devices_main(int argc, char *argv[])
{
px4_show_devices();
return 0;
}

static int list_topics_main(int argc, char *argv[])
{
px4_show_topics();
return 0;
}

static int list_files_main(int argc, char *argv[])
{
px4_show_files();
return 0;
}

static int sleep_main(int argc, char *argv[])
{
if (argc != 2) {
printf("Usage: sleep <seconds>\n");
return 1;
}
sleep(atoi(argv[1]));
return 0;
}

13 changes: 13 additions & 0 deletions cmake/posix/px4-alias.sh_in
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
#!/usr/bin/bash

# File is auto-generated by cmake compilation, do not edit.

# Ignore the expand_aliases command in zshell.
if [ ! -n "$ZSH_VERSION" ]; then
shopt -s expand_aliases
fi

# Don't stop on errors.
#set -e

${alias_string}
2 changes: 2 additions & 0 deletions launch/iris_env.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
#!/bin/bash
export PX4_SIM_MODEL="iris"
3 changes: 2 additions & 1 deletion launch/mavros_posix_sitl.launch
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
<arg name="vehicle" default="iris"/>
<arg name="world" default="$(find mavlink_sitl_gazebo)/worlds/empty.world"/>
<arg name="sdf" default="$(find mavlink_sitl_gazebo)/models/$(arg vehicle)/$(arg vehicle).sdf"/>
<arg name="rcS" default="$(find px4)/posix-configs/SITL/init/$(arg est)/$(arg vehicle)"/>
<arg name="rcS" default="$(find px4)/posix-configs/SITL/init/rcS"/>

<!-- gazebo configs -->
<arg name="gui" default="true"/>
<arg name="debug" default="false"/>
Expand Down
12 changes: 10 additions & 2 deletions launch/posix_sitl.launch
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,8 @@
<arg name="vehicle" default="iris"/>
<arg name="world" default="$(find mavlink_sitl_gazebo)/worlds/empty.world"/>
<arg name="sdf" default="$(find mavlink_sitl_gazebo)/models/$(arg vehicle)/$(arg vehicle).sdf"/>
<arg name="rcS" default="$(find px4)/posix-configs/SITL/init/$(arg est)/$(arg vehicle)"/>
<arg name="rcS" default="$(find px4)/posix-configs/SITL/init/rcS"/>

<!-- gazebo configs -->
<arg name="gui" default="true"/>
<arg name="debug" default="false"/>
Expand All @@ -26,7 +27,9 @@
<!-- PX4 SITL -->
<arg unless="$(arg interactive)" name="px4_command_arg1" value="-d"/>
<arg if="$(arg interactive)" name="px4_command_arg1" value=""/>
<node name="sitl" pkg="px4" type="px4" output="screen" args="$(find px4) $(arg rcS) $(arg px4_command_arg1)" required="true"/>
<node name="sitl" pkg="px4" type="px4" output="screen"
args="$(arg rootfs) $(arg rcS) $(arg px4_command_arg1)" />

<!-- Gazebo sim -->
<include file="$(find gazebo_ros)/launch/empty_world.launch">
<arg name="gui" value="$(arg gui)"/>
Expand All @@ -39,3 +42,8 @@
<!-- gazebo model -->
<node name="$(anon vehicle_spawn)" pkg="gazebo_ros" type="spawn_model" output="screen" args="-sdf -file $(arg sdf) -model $(arg vehicle) -x $(arg x) -y $(arg y) -z $(arg z) -R $(arg R) -P $(arg P) -Y $(arg Y)"/>
</launch>

<!-- This will set the environment variable needed to select iris in the startup. -->
<machine name="px4" env-loader="iris_env.sh" address="none" />
</launch>

1 change: 1 addition & 0 deletions msg/CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,7 @@ set(msg_files
power_button_state.msg
pwm_input.msg
qshell_req.msg
qshell_retval.msg
rate_ctrl_status.msg
rc_channels.msg
rc_parameter_map.msg
Expand Down
7 changes: 4 additions & 3 deletions msg/qshell_req.msg
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
int32[100] string
uint64 MAX_STRLEN = 100
uint64 strlen
uint8[100] cmd
uint32 MAX_STRLEN = 100
uint32 strlen
uint32 sequence
2 changes: 2 additions & 0 deletions msg/qshell_retval.msg
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
int32 return_value
uint32 sequence
Loading

0 comments on commit 0c5c741

Please sign in to comment.