[FL-2263] Flasher service & RAM exec (#1006)

* WIP on stripping fw * Compact FW build - use RAM_EXEC=1 COMPACT=1 DEBUG=0 * Fixed uninitialized storage struct; small fixes to compact fw * Flasher srv w/mocked flash ops * Fixed typos & accomodated FFF changes * Alternative fw startup branch * Working load & jmp to RAM fw * +manifest processing for stage loader; + crc verification for stage payload * Fixed questionable code & potential leaks * Lowered screen update rate; added radio stack update stubs; working dfu write * Console EP with manifest & stage validation * Added microtar lib; minor ui fixes for updater * Removed microtar * Removed mtar Flipper-XFW#2 * Added a better version of microtar * TAR archive api; LFS backup & restore core * Recursive backup/restore * LFS worker thread * Added system apps to loader - not visible in UI; full update process with restarts * Typo fix * Dropped BL & f6; tooling for updater WIP * Minor py fixes * Minor fixes to make it build after merge * Ported flash workaround from BL + fixed visuals * Minor cleanup * Chmod + loader app search fix * Python linter fix * Removed usb stuff & float read support for staged loader == -10% of binary size * Added backup/restore & update pb requests * Added stub impl to RPC for backup/restore/update commands * Reworked TAR to use borrowed Storage api; slightly reduced build size by removing `static string`; hidden update-related RPC behind defines * Moved backup&restore to storage * Fixed new message types * Backup/restore/update RPC impl * Moved furi_hal_crc to LL; minor fixes * CRC HAL rework to LL * Purging STM HAL * Brought back minimal DFU boot mode (no gui); additional crc state checks * Added splash screen, BROKEN usb function * Clock init rework WIP * Stripped graphics from DFU mode * Temp fix for unused static fun * WIP update picker - broken! * Fixed UI * Bumping version * Fixed RTC setup * Backup to update folder instead of ext root * Removed unused scenes & more usb remnants from staged loader * CI updates * Fixed update bundle name * Temporary restored USB handler * Attempt to prevent .text corruption * Comments on how I spent this Saturday * Added update file icon * Documentation updates * Moved common code to lib folder * Storage: more unit tests * Storage: blocking dir open, differentiate file and dir when freed. * Major refactoring; added input processing to updater to allow retrying on failures (not very useful prob). Added API for extraction of thread return value * Removed re-init check for manifest * Changed low-level path manipulation to toolbox/path.h; makefile cleanup; tiny fix in lint.py * Increased update worker stack size * Text fixes in backup CLI * Displaying number of update stages to run; removed timeout in handling errors * Bumping version * Added thread cleanup for spawner thread * Updated build targets to exclude firmware bundle from 'ALL' * Fixed makefile for update_package; skipping VCP init for update mode (ugly) * Switched github build from ALL to update_package * Added +x for dist_update.sh * Cli: add total heap size to "free" command * Moved (RAM) suffix to build version instead of git commit no. * DFU comment * Some fixes suggested by clang-tidy * Fixed recursive PREFIX macro * Makefile: gather all new rules in updater namespace. FuriHal: rename bootloader to boot, isr safe delays * Github: correct build target name in firmware build * FuriHal: move target switch to boot * Makefile: fix firmware flash * Furi, FuriHal: move kernel start to furi, early init * Drop bootloader related stuff * Drop cube. Drop bootloader linker script. * Renamed update_hl, moved constants to #defines * Moved update-related boot mode to separate bitfield * Reworked updater cli to single entry point; fixed crash on tar cleanup * Added Python replacement for dist shell scripts * Linter fixes for dist.py +x * Fixes for environment suffix * Dropped bash scripts * Added dirty build flag to version structure & interfaces * Version string escapes * Fixed flag logic in dist.py; added support for App instances being imported and not terminating the whole program * Fixed fw address in ReadMe.md * Rpc: fix crash on double screen start * Return back original boot behavior and fix jump to system bootloader * Cleanup code, add error sequence for RTC * Update firmware readme * FuriHal: drop boot, restructure RTC registers usage and add header register check * Furi goes first * Toolchain: add ccache support * Renamed update bundle dir Co-authored-by: DrZlo13 <[email protected]> Co-authored-by: あく <[email protected]>
tori-ham · Apr 13, 2022 · e020401 · e020401
1 parent a25552e
commit e020401
Show file tree

Hide file tree

Showing 221 changed files with 4,194 additions and 11,699 deletions.
diff --git a/.github/CODEOWNERS b/.github/CODEOWNERS
@@ -25,6 +25,7 @@ applications/sd-filesystem/** @skotopes @DrZlo13
 applications/subghz/** @skotopes @DrZlo13
 applications/template/** @skotopes @DrZlo13
 applications/tests/** @skotopes @DrZlo13
+applications/updater/** @skotopes @DrZlo13 @hedger
 
 # Assets and asset generator
 assets/** @skotopes @DrZlo13

diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml
@@ -42,7 +42,7 @@ jobs:
         run: |
           test -d artifacts && rm -rf artifacts || true
           mkdir artifacts
-      
+
       - name: 'Generate suffix and folder name'
         id: names
         run: |
@@ -52,13 +52,13 @@ jobs:
           fi
           BRANCH_OR_TAG=${REF#refs/*/}
           SHA=$(git rev-parse --short HEAD)
-          
+
           if [[ "${{ github.ref }}" == "refs/tags/"* ]]; then
             SUFFIX=${BRANCH_OR_TAG//\//_}
           else
             SUFFIX=${BRANCH_OR_TAG//\//_}-$(date +'%d%m%Y')-${SHA}
           fi
-          
+
           echo "WORKFLOW_BRANCH_OR_TAG=${BRANCH_OR_TAG}" >> $GITHUB_ENV
           echo "DIST_SUFFIX=${SUFFIX}" >> $GITHUB_ENV
           echo "::set-output name=artifacts-path::${BRANCH_OR_TAG}"
@@ -86,7 +86,7 @@ jobs:
             set -e
             for TARGET in ${TARGETS}
             do
-              make TARGET=${TARGET} ${{ startsWith(github.ref, 'refs/tags') && 'DEBUG=0 COMPACT=1' || '' }}
+              make updater_package TARGET=${TARGET} ${{ startsWith(github.ref, 'refs/tags') && 'DEBUG=0 COMPACT=1' || '' }}
             done
 
       - name: 'Move upload files'
@@ -100,6 +100,17 @@ jobs:
               mv dist/${TARGET}/* artifacts/
             done
 
+      - name: 'Bundle self-update package'
+        if: ${{ !github.event.pull_request.head.repo.fork }}
+        run: |
+          set -e
+          for UPDATEBUNDLE in artifacts/*/
+          do
+            BUNDLE_NAME=`echo $UPDATEBUNDLE | cut -d'/' -f2`
+            echo Packaging ${BUNDLE_NAME}
+            tar czpf artifacts/flipper-z-${BUNDLE_NAME}.tgz -C artifacts ${BUNDLE_NAME}
+          done
+
       - name: 'Bundle resources'
         if: ${{ !github.event.pull_request.head.repo.fork }}
         run: |
@@ -175,7 +186,7 @@ jobs:
 
       - name: 'Build docker image'
         uses: ./.github/actions/docker
-      
+
       - name: 'Generate suffix and folder name'
         id: names
         run: |
@@ -185,13 +196,13 @@ jobs:
           fi
           BRANCH_OR_TAG=${REF#refs/*/}
           SHA=$(git rev-parse --short HEAD)
-          
+
           if [[ "${{ github.ref }}" == "refs/tags/"* ]]; then
             SUFFIX=${BRANCH_OR_TAG//\//_}
           else
             SUFFIX=${BRANCH_OR_TAG//\//_}-$(date +'%d%m%Y')-${SHA}
           fi
-          
+
           echo "WORKFLOW_BRANCH_OR_TAG=${BRANCH_OR_TAG}" >> $GITHUB_ENV
           echo "DIST_SUFFIX=${SUFFIX}" >> $GITHUB_ENV
 

diff --git a/.gitmodules b/.gitmodules
@@ -19,3 +19,6 @@
 [submodule "lib/FreeRTOS-Kernel"]
 	path = lib/FreeRTOS-Kernel
 	url = https://github.com/FreeRTOS/FreeRTOS-Kernel.git
+[submodule "lib/microtar"]
+	path = lib/microtar
+	url = https://github.com/amachronic/microtar.git
diff --git a/Makefile b/Makefile
@@ -1,16 +1,16 @@
 PROJECT_ROOT := $(abspath $(dir $(abspath $(firstword $(MAKEFILE_LIST)))))
+
+include			$(PROJECT_ROOT)/make/git.mk
+
 COPRO_DIR := $(PROJECT_ROOT)/lib/STM32CubeWB/Projects/STM32WB_Copro_Wireless_Binaries/STM32WB5x
 
 PROJECT_SOURCE_DIRECTORIES := \
 	$(PROJECT_ROOT)/applications \
-	$(PROJECT_ROOT)/bootloader/src \
-	$(PROJECT_ROOT)/bootloader/targets \
 	$(PROJECT_ROOT)/core \
 	$(PROJECT_ROOT)/firmware/targets \
 	$(PROJECT_ROOT)/lib/app-template \
 	$(PROJECT_ROOT)/lib/app-scened-template \
 	$(PROJECT_ROOT)/lib/common-api \
-	$(PROJECT_ROOT)/lib/cyfral \
 	$(PROJECT_ROOT)/lib/drivers \
 	$(PROJECT_ROOT)/lib/flipper_file \
 	$(PROJECT_ROOT)/lib/infrared \
@@ -34,18 +34,18 @@ endif
 include	$(PROJECT_ROOT)/make/defaults.mk
 
 .PHONY: all
-all: bootloader_all firmware_all
-	@$(PROJECT_ROOT)/scripts/dist.sh
+all: firmware_all
+	@$(PROJECT_ROOT)/scripts/dist.py copy -t $(TARGET) -p firmware -s $(DIST_SUFFIX)
 
 .PHONY: whole
-whole: flash_radio bootloader_flash firmware_flash
+whole: flash_radio firmware_flash
 
 .PHONY: clean
-clean: bootloader_clean firmware_clean
+clean: firmware_clean updater_clean
 	@rm -rf $(PROJECT_ROOT)/dist/$(TARGET)
 
 .PHONY: flash
-flash: bootloader_flash firmware_flash
+flash: firmware_flash
 
 .PHONY: debug
 debug:
@@ -60,36 +60,38 @@ wipe:
 	@$(PROJECT_ROOT)/scripts/flash.py wipe
 	@$(PROJECT_ROOT)/scripts/ob.py set
 
-.PHONY: bootloader_all
-bootloader_all:
-	@$(MAKE) -C $(PROJECT_ROOT)/bootloader -j$(NPROCS) all
-
 .PHONY: firmware_all
 firmware_all:
 	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) all
 
-.PHONY: bootloader_clean
-bootloader_clean:
-	@$(MAKE) -C $(PROJECT_ROOT)/bootloader -j$(NPROCS) clean
-
 .PHONY: firmware_clean
 firmware_clean:
 	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) clean
 
-.PHONY: bootloader_flash
-bootloader_flash:
-ifeq ($(FORCE), 1)
-	@rm $(PROJECT_ROOT)/bootloader/.obj/f*/flash || true
-endif
-	@$(MAKE) -C $(PROJECT_ROOT)/bootloader -j$(NPROCS) flash
-
 .PHONY: firmware_flash
 firmware_flash:
 ifeq ($(FORCE), 1)
-	@rm $(PROJECT_ROOT)/firmware/.obj/f*/flash || true
+	@rm $(PROJECT_ROOT)/firmware/.obj/f*-firmware/flash || true
 endif
 	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) flash
 
+
+.PHONY: updater
+updater:
+	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) RAM_EXEC=1 all
+
+.PHONY: updater_clean
+updater_clean:
+	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) RAM_EXEC=1 clean
+
+.PHONY: updater_debug
+updater_debug:
+	@$(MAKE) -C $(PROJECT_ROOT)/firmware -j$(NPROCS) RAM_EXEC=1 debug
+
+.PHONY: updater_package
+updater_package: firmware_all updater
+	@$(PROJECT_ROOT)/scripts/dist.py copy -t $(TARGET) -p firmware updater -s $(DIST_SUFFIX) --bundlever "$(VERSION_STRING)"
+
 .PHONY: flash_radio
 flash_radio:
 	@$(PROJECT_ROOT)/scripts/flash.py core2radio 0x080D7000 $(COPRO_DIR)/stm32wb5x_BLE_Stack_light_fw.bin
@@ -100,7 +102,7 @@ flash_radio_fus:
 	@echo
 	@echo "================   DON'T DO IT    ================"
 	@echo "= Flashing FUS is going to erase secure enclave  ="
-	@echo "= You will loose ability to use encrypted assets ="
+	@echo "= You will lose ability to use encrypted assets  ="
 	@echo "=       type 'find / -exec rm -rf {} \;'         ="
 	@echo "=     In case if you still want to continue      ="
 	@echo "================    JUST DON'T    ================"

diff --git a/ReadMe.md b/ReadMe.md
@@ -11,17 +11,16 @@ Our goal is to create nice and clean code with good documentation, to make it a
 
 [Get Latest Firmware from Update Server](https://update.flipperzero.one/)
 
-Flipper Zero's firmware consists of three components:
+Flipper Zero's firmware consists of two components:
 
 - Core2 firmware set - proprietary components by ST: FUS + radio stack. FUS is flashed at factory and you should never update it.
-- Core1 Bootloader - controls basic hardware initialization and loads firmware.
 - Core1 Firmware - HAL + OS + Drivers + Applications.
 
 All 3 of them must be flashed in order described.
 
 ## With STLink
 
-### Core1 Bootloader + Firmware
+### Core1 Firmware
 
 Prerequisites:
 
@@ -144,7 +143,6 @@ make whole
 
 - `applications`    - Applications and services used in firmware
 - `assets`          - Assets used by applications and services
-- `bootloader`      - Bootloader source code
 - `core`            - Furi Core: os level primitives and helpers
 - `debug`           - Debug tool: GDB-plugins, SVD-file and etc
 - `docker`          - Docker image sources (used for firmware build automation)

diff --git a/applications/ReadMe.md b/applications/ReadMe.md
@@ -34,6 +34,7 @@
 - `system`              - System settings, tools and API
 - `tests`               - Unit tests and etc
 - `u2f`                 - U2F Application
+- `updater`             - Update service & application
 
 - `application.c`       - Firmware application list source
 - `application.h`       - Firmware application list header

diff --git a/applications/about/about.c b/applications/about/about.c
@@ -116,9 +116,10 @@ static DialogMessageButton fw_version_screen(DialogsApp* dialogs, DialogMessage*
     } else {
         string_cat_printf(
             buffer,
-            "%s [%s]\n%s [%s]\n[%d] %s",
+            "%s [%s]\n%s%s [%s]\n[%d] %s",
             version_get_version(ver),
             version_get_builddate(ver),
+            version_get_dirty_flag(ver) ? "[!] " : "",
             version_get_githash(ver),
             version_get_gitbranchnum(ver),
             version_get_target(ver),
@@ -135,45 +136,14 @@ static DialogMessageButton fw_version_screen(DialogsApp* dialogs, DialogMessage*
     return result;
 }
 
-static DialogMessageButton bootloader_version_screen(DialogsApp* dialogs, DialogMessage* message) {
-    DialogMessageButton result;
-    string_t buffer;
-    string_init(buffer);
-    const Version* ver = furi_hal_version_get_bootloader_version();
-
-    if(!ver) {
-        string_cat_printf(buffer, "No info\n");
-    } else {
-        string_cat_printf(
-            buffer,
-            "%s [%s]\n%s [%s]\n[%d] %s",
-            version_get_version(ver),
-            version_get_builddate(ver),
-            version_get_githash(ver),
-            version_get_gitbranchnum(ver),
-            version_get_target(ver),
-            version_get_gitbranch(ver));
-    }
-
-    dialog_message_set_header(message, "Boot Version info:", 0, 0, AlignLeft, AlignTop);
-    dialog_message_set_text(message, string_get_cstr(buffer), 0, 13, AlignLeft, AlignTop);
-    result = dialog_message_show(dialogs, message);
-    dialog_message_set_text(message, NULL, 0, 0, AlignLeft, AlignTop);
-    dialog_message_set_header(message, NULL, 0, 0, AlignLeft, AlignTop);
-    string_clear(buffer);
-
-    return result;
-}
-
 const AboutDialogScreen about_screens[] = {
     product_screen,
     compliance_screen,
     address_screen,
     icon1_screen,
     icon2_screen,
     hw_version_screen,
-    fw_version_screen,
-    bootloader_version_screen};
+    fw_version_screen};
 
 const size_t about_screens_count = sizeof(about_screens) / sizeof(AboutDialogScreen);
 

diff --git a/applications/applications.c b/applications/applications.c
@@ -14,6 +14,7 @@ extern int32_t notification_srv(void* p);
 extern int32_t power_srv(void* p);
 extern int32_t storage_srv(void* p);
 extern int32_t desktop_srv(void* p);
+extern int32_t updater_srv(void* p);
 
 // Apps
 extern int32_t accessor_app(void* p);
@@ -58,6 +59,7 @@ extern void storage_on_system_start();
 extern void subghz_on_system_start();
 extern void power_on_system_start();
 extern void unit_tests_on_system_start();
+extern void updater_on_system_start();
 
 // Settings
 extern int32_t notification_settings_app(void* p);
@@ -91,6 +93,9 @@ const FlipperApplication FLIPPER_SERVICES[] = {
 #endif
 
 #ifdef SRV_DESKTOP
+#ifdef SRV_UPDATER
+#error SRV_UPDATER and SRV_DESKTOP are mutually exclusive!
+#endif
     {.app = desktop_srv, .name = "DesktopSrv", .stack_size = 2048, .icon = NULL},
 #endif
 
@@ -117,10 +122,28 @@ const FlipperApplication FLIPPER_SERVICES[] = {
 #ifdef SRV_STORAGE
     {.app = storage_srv, .name = "StorageSrv", .stack_size = 3072, .icon = NULL},
 #endif
+
+#ifdef SRV_UPDATER
+#ifdef SRV_DESKTOP
+#error SRV_UPDATER and SRV_DESKTOP are mutually exclusive!
+#endif
+    {.app = updater_srv, .name = "UpdaterSrv", .stack_size = 2048, .icon = NULL},
+#endif
 };
 
 const size_t FLIPPER_SERVICES_COUNT = sizeof(FLIPPER_SERVICES) / sizeof(FlipperApplication);
 
+const FlipperApplication FLIPPER_SYSTEM_APPS[] = {
+#ifdef APP_UPDATER
+#ifdef SRV_UPDATER
+#error APP_UPDATER and SRV_UPDATER are mutually exclusive!
+#endif
+    {.app = updater_srv, .name = "UpdaterApp", .stack_size = 2048, .icon = NULL},
+#endif
+};
+
+const size_t FLIPPER_SYSTEM_APPS_COUNT = sizeof(FLIPPER_SERVICES) / sizeof(FlipperApplication);
+
 // Main menu APP
 const FlipperApplication FLIPPER_APPS[] = {
 
@@ -199,6 +222,10 @@ const FlipperOnStartHook FLIPPER_ON_SYSTEM_START[] = {
 #ifdef APP_UNIT_TESTS
     unit_tests_on_system_start,
 #endif
+
+#ifdef APP_UPDATER
+    updater_on_system_start,
+#endif
 };
 
 const size_t FLIPPER_ON_SYSTEM_START_COUNT =
@@ -326,4 +353,4 @@ const FlipperApplication FLIPPER_SETTINGS_APPS[] = {
 };
 
 const size_t FLIPPER_SETTINGS_APPS_COUNT =
-    sizeof(FLIPPER_SETTINGS_APPS) / sizeof(FlipperApplication);
+    sizeof(FLIPPER_SETTINGS_APPS) / sizeof(FlipperApplication);
diff --git a/applications/applications.h b/applications/applications.h
@@ -42,6 +42,12 @@ extern const size_t FLIPPER_PLUGINS_COUNT;
 extern const FlipperApplication FLIPPER_DEBUG_APPS[];
 extern const size_t FLIPPER_DEBUG_APPS_COUNT;
 
+/* System apps
+ * Can only be spawned by loader by name
+ */
+extern const FlipperApplication FLIPPER_SYSTEM_APPS[];
+extern const size_t FLIPPER_SYSTEM_APPS_COUNT;
+
 /* Seperate scene app holder
  * Spawned by loader
  */
@@ -55,4 +61,4 @@ extern const FlipperApplication FLIPPER_ARCHIVE;
  * Spawned by loader
  */
 extern const FlipperApplication FLIPPER_SETTINGS_APPS[];
-extern const size_t FLIPPER_SETTINGS_APPS_COUNT;
+extern const size_t FLIPPER_SETTINGS_APPS_COUNT;