Updated dependencies

Author: jerzyKleszcz

FileExtractor/README.md

@@ -11,7 +11,7 @@ The information include the list of files containing the Swift source code that
 ## Usage
 
 ```bash
-$ file-extractor -projectrootpath <path-to-xcode-project> -filesjson <path-to-output-file>
+$ file-extractor -projectrootpath <path-to-xcode-project> [-filesjson <path-to-output-file>] [-projectfile <path-to-xcodeproj>]
 ```
 
 where
@@ -20,6 +20,8 @@ where
 
 `<path-to-output-file>` is a path to the file that the extraced data will be written to. If it's an optional parameter. If ommited, tool will print out to the standard output.
 
+`<path-to-xcodeproj>` is a path to the Xcode project file. It's an optional parameter and should be provided only when the tool fails to automatically identify which project to parse.
+
 ## Data formats
 
 The input data format is defined by the Xcode project file structure and best described [here](http://www.monobjc.net/xcode-project-file-format.html). The user should not need to ever work with it outside of Xcode.
@@ -42,13 +44,14 @@ The output data format is called `Files.json` and presented below:
   },
   "sourceFiles": [ <string> ],
   "layoutFiles": [ <string> ],
-  "systemLinkedFrameworks": [<string>],
+  "implicitlyLinkedFrameworks": [ <string> ],
   "explicitlyLinkedFrameworks": [
     {
       "name": <string>,
       "path": <string>
     }
-  ]
+  ],
+  "frameworkSearchPaths": [ <string> ]
 }
 ```
 `project` is an object that contains the path to the project root directory and path to the actual xcodeproj or xcworkspace file. This directory will be copied by the Renamer to provide place for writing the obfuscated Swift source files to.
@@ -61,10 +64,12 @@ The output data format is called `Files.json` and presented below:
 
 `layoutFiles` is an array of paths to storyboard / xib files containing the layouts that the tool should obfuscate.
 
-`systemLinkedFrameworks` contains the list of names of frameworks that are imported in the source code (with various form of Swift `import` statement), but not included in the Xcode project. Since Xcode autolinks the system frameworks by default (see `CLANG_MODULES_AUTOLINK` flag), these frameworks should be automatically found by the compiler, which uses the SDK path for this purpose. They are used to identify the module that the symbol is part of.
+`implicitlyLinkedFrameworks` contains the list of names of frameworks that are imported in the source code (with various form of Swift `import` statement), but not included in the Xcode project. There are two types of such frameworks. One is system frameworks. Since Xcode autolinks the system frameworks by default (see `CLANG_MODULES_AUTOLINK` flag), these frameworks should be automatically found by the compiler, which uses the SDK path for this purpose. Second one is frameworks with paths added to search paths, but not explicitely stated as dependencies. They are used to identify the module that the symbol is part of.
 
 `explicitlyLinkedFrameworks` contains the list of framework objects with name and path. These are taken from the Xcode project, which must contain the names and paths to frameworks that are not automatically linked. They are required for the Swift compiler to perform the analysis and also used to identify which module is the symbol part of.
 
+`frameworkSearchPaths` contains the list of paths to search for non-system frameworks. It's especially useful when working with Cocoapods projects, since they dont specify the framework dependencies explicitely, but set the paths.
+
 Sample `Files.json` file might look like that:
 
 ```javascript
@@ -95,8 +100,11 @@ Sample `Files.json` file might look like that:
          "path":"/Applications/Xcode.app/Contents/Developer/Platforms/iPhoneOS.platform/Developer/SDKs/iPhoneOS11.0.sdk/System/Library/Frameworks/"
       }
    ],
-   "systemLinkedFrameworks":[
+   "implicitlyLinkedFrameworks":[
       "UIKit"
+   ],
+   "frameworkSearchPaths": [
+      "/Users/siejkowski/Polidea/SwiftObfuscator/TestProjects/Dropnote/Pods/Crashlytics/iOS"
    ]
 }
 ```

FileExtractor/lib/file-extractor/command.rb

@@ -3,8 +3,10 @@ module FileExtractor
   require 'claide'
 
   require 'file-extractor/arguments_decorator'
-  require 'file-extractor/xcodeproj_determiner'
   require 'file-extractor/data_extractor'
+  require 'file-extractor/dependency_builder'
+  require 'file-extractor/xcodefiles_determiner'
+  require 'file-extractor/xcworkspace_extractor'
 
   class Command < CLAide::Command
 
@@ -13,13 +15,17 @@ class Command < CLAide::Command
 
     PROJECTROOTPATH_KEY = "projectrootpath"
     FILESJSON_KEY = "filesjson"
+    PROJECTFILEPATH_KEY = "projectfile"
 
     @file_extractor_options = [
       ["-#{PROJECTROOTPATH_KEY}", "Path to Xcode project root folder. "\
                         "It\'s the folder that contains both the Xcode project file (.xcodeproj or .xcworkspace) "\
                         "and the source files."\
                         "It\'s a required parameter."],
-      ["-#{FILESJSON_KEY}", "Path to output files.json. It\'s an optional parameter."]
+      ["-#{FILESJSON_KEY}", "Path to output files.json. It\'s an optional parameter."],
+      ["-#{PROJECTFILEPATH_KEY}", "is a path to the Xcode project file. "\
+                                  "It\'s an optional parameter and should be provided only "\
+                                  "when the tool fails to automatically identify which project to parse."]
     ]
 
     def self.options
@@ -31,7 +37,9 @@ def self.options
       CLAide::Argument.new("-#{PROJECTROOTPATH_KEY}", true),
       CLAide::Argument.new("PROJECTROOT", true),
       CLAide::Argument.new("-#{FILESJSON_KEY}", false),
-      CLAide::Argument.new("FILESJSON", false)
+      CLAide::Argument.new("FILESJSON", false),
+      CLAide::Argument.new("-#{PROJECTFILEPATH_KEY}", false),
+      CLAide::Argument.new("PROJECTFILE", false)
     ]
 
     def extract_option(argv, key)
@@ -46,6 +54,7 @@ def extract_option(argv, key)
     def initialize(argv)
       @project_root_path = extract_option(argv, PROJECTROOTPATH_KEY)
       @files_path = extract_option(argv, FILESJSON_KEY)
+      @project_file_path = extract_option(argv, PROJECTFILEPATH_KEY)
       super
     end
 
@@ -57,24 +66,42 @@ def self.run(argv = [])
     def run
       puts "Path to root:"
       puts @project_root_path
-      project_paths = FileExtractor::XcodeprojDeterminer.find_xcode_files(@project_root_path)
-      if project_paths.empty?
+      xcodeprojs, xcworkspaces = FileExtractor::XcodefilesDeterminer.find_xcode_files(@project_root_path)
+      projects, schemes = FileExtractor::XcworkspaceExtractor.extract_projects_and_dependency_schemes(xcworkspaces, xcodeprojs)
+      if projects.empty?
         puts "\n\nNo Xcode project document found in the project root directory"
-      elsif project_paths.count == 1
+      elsif projects.count == 1
         puts "Path to Xcode project:"
-        puts project_paths.first
+        puts xcodeprojs.first
         puts "Path to files:"
         puts @files_path
 
-        json, output_string = FileExtractor::DataExtractor.extract_data(@project_root_path, project_paths.first, @files_path)
+        json, output_string, build_dir = FileExtractor::DataExtractor.extract_data(@project_root_path, xcodeprojs.first, @files_path)
         puts "\nFound data:\n#{json}"
         puts "\n#{output_string}"
-      else 
+        
+        FileExtractor::DependencyBuilder.build_dependencies(schemes, build_dir)
+      elsif !@project_file_path.nil? 
+        puts "Path to Xcode project:"
+        puts @project_file_path
+        puts "Path to files:"
+        puts @files_path
+
+        json, output_string, build_dir = FileExtractor::DataExtractor.extract_data(@project_root_path, xcodeprojs.first, @files_path)
+        puts "\nFound data:\n#{json}"
+        puts "\n#{output_string}"
+
+        if projects.include? @project_file_path
+          FileExtractor::DependencyBuilder.build_dependencies(schemes, build_dir)
+        end
+      else
         puts "\n\nFound multiple possible Xcode project files:\n"
-        project_paths.each do |path|
+        projects.each do |path|
           puts path
         end
+        puts "\nPlease specify which one to use with -#{PROJECTFILEPATH_KEY} flag\n"
       end
     end
+
   end
 end

FileExtractor/lib/file-extractor/data_extractor.rb

@@ -3,31 +3,28 @@ module FileExtractor
   require 'file-extractor/files_json_struct'
   require 'file-extractor/xcodeproj_extractor'
   require 'file-extractor/modules_extractor'
-  require 'file-extractor/sdk_resolver'
   require 'file-extractor/data_integrity_checker'
 
   class DataExtractor
 
     def self.extract_data(root_path, project_path, files_path)      
-      data = extract_data_from_all_sources(root_path, project_path)
+      data, build_dir = extract_data_from_all_sources(root_path, project_path)
       data_is_ok, error_message = FileExtractor::DataIntegrityChecker.verify_data_integrity(data)
       json = JSON.pretty_generate(data)
       if data_is_ok 
-        return json, write_json_to_file(json, files_path)
+        return json, write_json_to_file(json, files_path), build_dir
       else
-        return json, error_message
+        return json, error_message, build_dir
       end
     end
 
     private
 
     def self.extract_data_from_all_sources(root_path, project_path)
       data_extractor = FileExtractor::XcodeprojExtractor.new(root_path, project_path)
-      data = data_extractor.extract_data
-      data.systemLinkedFrameworks = FileExtractor::ModulesExtractor.system_linked_frameworks(data)
-      data.sdk.path = FileExtractor::SdkResolver.sdk_path(data)
-      data.explicitlyLinkedFrameworks = FileExtractor::SdkResolver.update_frameworks_paths(data)
-      data
+      data, build_dir = data_extractor.extract_data_and_build_directory
+      data.implicitlyLinkedFrameworks = FileExtractor::ModulesExtractor.system_linked_frameworks(data)
+      return data, build_dir
     end
 
     def self.write_json_to_file(json, files_path)

FileExtractor/lib/file-extractor/dependency_builder.rb

@@ -0,0 +1,41 @@
+module FileExtractor
+
+  require 'open3'
+
+  class DependencyBuilder
+
+    def self.build_dependencies(schemes, build_dir)
+      schemes.each do |scheme|
+        puts "\nBuilding dependencies using command:\n"
+        build_dir = clean_build_directory(build_dir)
+        clean_command = "xcodebuild -workspace \"#{scheme[:workspace]}\" -scheme \"#{scheme[:scheme]}\" -derivedDataPath \"#{build_dir}\" clean"
+        build_command = "xcodebuild -workspace \"#{scheme[:workspace]}\" -scheme \"#{scheme[:scheme]}\" -derivedDataPath \"#{build_dir}\""
+        puts clean_command
+        puts build_command
+        Open3.capture3(clean_command)
+        stdout, stderr, status = Open3.capture3(build_command)
+        if status.success?
+          puts "\nBuilding dependencies finished successfully\n"
+        else
+          puts "\nError while building dependencies, here come the logs\n\n"
+          puts "\nstdout:\n\n"
+          puts stdout
+          puts "\nstderr:\n\n"
+          puts stderr
+        end
+      end
+    end
+
+    def self.clean_build_directory(build_dir)
+      # TODO: should we pass the configuration names from build settings here instead of "Debug" and "Release"?
+      paths_to_remove = ["Build", "Products", "Debug", "Release"]
+      dirs = build_dir.split(File::SEPARATOR)
+      while dirs.last.start_with?(*paths_to_remove)
+        dirs.pop
+      end
+      File.expand_path(dirs.join(File::SEPARATOR))
+    end
+
+  end
+
+end

FileExtractor/lib/file-extractor/files_json_struct.rb

@@ -24,7 +24,14 @@ def to_json(options = {})
     include StructSerialization
   end
 
-  FilesJson = Struct.new(:project, :module, :sdk, :sourceFiles, :layoutFiles, :explicitlyLinkedFrameworks, :systemLinkedFrameworks) do
+  FilesJson = Struct.new(:project, 
+                         :module, 
+                         :sdk, 
+                         :sourceFiles, 
+                         :layoutFiles, 
+                         :explicitlyLinkedFrameworks, 
+                         :implicitlyLinkedFrameworks, 
+                         :frameworkSearchPaths) do
     include StructSerialization
 
     def to_h

FileExtractor/lib/file-extractor/xcodefiles_determiner.rb

@@ -0,0 +1,39 @@
+module FileExtractor
+
+  require 'find'
+
+  class XcodefilesDeterminer
+
+    XCODE_PROJECT_FORMAT = ".xcodeproj"
+    XCWORKSPACE_FORMAT = ".xcworkspace"
+    XCODE_FILE_FORMATS = [XCODE_PROJECT_FORMAT, XCWORKSPACE_FORMAT]
+
+    def self.find_xcode_files(path)
+      xcodeprojs = find_xcode_files_of_type(path, XCODE_PROJECT_FORMAT)
+      xcworkspaces = find_xcode_files_of_type(path, XCWORKSPACE_FORMAT)
+      return xcodeprojs, xcworkspaces
+    end
+
+    private 
+
+    def self.find_xcode_files_of_type(path, type)
+      Find.find(path).select do |path| 
+        check_if_path_has_extension(path, type)
+      end.each do |path|
+        File.expand_path(path)
+      end
+    end
+
+    def self.check_if_path_has_extension(path, extension)
+      extension == File.extname(path) && check_if_is_not_subfile(path)
+    end
+
+    def self.check_if_is_not_subfile(file)
+      XCODE_FILE_FORMATS.select do |extension| 
+        File.dirname(file).include?(extension)
+      end.empty?
+    end
+
+  end
+
+end

FileExtractor/lib/file-extractor/xcodeproj_extractor.rb

@@ -15,26 +15,32 @@ def initialize(root_path, project_path)
       @main_build_settings = build_settings(@main_target)
     end
 
-    def extract_data
-      FileExtractor::FilesJson.new(
+    def extract_data_and_build_directory
+      dir = @main_build_settings["BUILT_PRODUCTS_DIR"]
+      return FileExtractor::FilesJson.new(
         FileExtractor::Project.new(@root_path, @project.path.to_s),
-        FileExtractor::Module.new(module_name(@main_target), triple(@main_build_settings)),
-        FileExtractor::Sdk.new(sdk(@main_target), nil),
+        FileExtractor::Module.new(module_name(@main_build_settings), triple(@main_build_settings)),
+        FileExtractor::Sdk.new(sdk_name(@main_build_settings), sdk_path(@main_build_settings)),
         source_files(@main_target),
         layout_files(@main_target),
-        explicitly_linked_frameworks(@main_target),
-        nil
-      )
+        update_frameworks_paths(explicitly_linked_frameworks(@main_target), @main_build_settings),
+        nil,
+        framework_search_paths(@main_build_settings)
+      ), dir
     end
 
     private
 
-    def module_name(target)
-      target.name
+    def module_name(build_settings)
+      build_settings["PRODUCT_MODULE_NAME"]
     end
 
-    def sdk(target)
-      target.sdk
+    def sdk_name(build_settings)
+      build_settings["SDK_NAME"]
+    end
+
+    def sdk_path(build_settings)
+      build_settings["SDKROOT"]
     end
 
     def source_files(target)
@@ -65,14 +71,36 @@ def filepaths(file_references, *extensions)
 
     def explicitly_linked_frameworks(target)
       target.frameworks_build_phase.files.map do |framework|
-        name = framework.file_ref.name
+        name = framework.display_name
         path = framework.file_ref.real_path.to_s
         FileExtractor::ExplicitlyLinkedFramework.new(name.sub(".framework", ""), path.sub(name, ""))
       end
     end
 
+    def update_frameworks_paths(frameworks, build_settings)
+      frameworks.map do |framework|
+        framework.path = framework.path.sub("${SDKROOT}", build_settings["SDKROOT"])
+        framework.path = framework.path.sub("${BUILT_PRODUCTS_DIR}", build_settings["BUILT_PRODUCTS_DIR"])
+        framework
+      end
+    end
+
+    def framework_search_paths(build_settings)
+      search_paths = build_settings["FRAMEWORK_SEARCH_PATHS"]
+      framework_search_paths = []
+      if !search_paths.nil?
+        framework_search_paths = search_paths.split("\" ").map do |path_to_clean|
+          path_to_clean.sub(/^\"/, '')
+        end
+      end
+      framework_search_paths
+    end
+
     def triple(build_settings)
-      architecture = build_settings["CURRENT_ARCH"]
+      architecture = build_settings["PLATFORM_PREFERRED_ARCH"]
+      if architecture.nil?
+        architecture = build_settings["CURRENT_ARCH"]
+      end
       sdk = case build_settings["PLATFORM_NAME"]
         when "iphoneos"
           build_settings["SDK_NAME"].gsub("iphoneos", "ios")