populse_mia.user_interface.pipeline_manager package

Handle the pipeline manager tab. Allow to create and run pipelines.

Contains:

Module:

• iteration_table

• node_controller

• pipeline_editor

• pipeline_manager_tab

• process_library

• process_mia

Submodules

populse_mia.user_interface.pipeline_manager.iteration_table module

Module that handles pipeline iteration.

Contains:

Class:

• IterationTable

class populse_mia.user_interface.pipeline_manager.iteration_table.IterationTable(project, scan_list, main_window)

Bases: QWidget

Widget that handles pipeline iteration.

__init__(project, scan_list, main_window)

Initialization of the IterationTable widget.

Parameters:

• project – current project in the software

• scan_list – list of the selected database files

• main_window – software’s main_window

add_tag()

Add a tag to visualize in the iteration table.

Used only for tests

emit_iteration_table_updated()

Emit a signal when the iteration scans have been updated.

fill_values(idx)

Fill values_list depending on the visualized tags

Parameters:

idx – Index of the tag

filter_values()

blabla

iteration_table_updated

refresh_layout()

Update the layout of the widget.

Called in widget’s initialization and when a tag push button is added or removed.

remove_tag()

Remove a tag to visualize in the iteration table.

select_iteration_tag()

Open a pop-up to let the user select on which tag to iterate.

select_visualized_tag(idx)

Open a pop-up to let the user select which tag to visualize in the iteration table.

Parameters:

idx – index of the clicked push button

update_iterated_tag(tag_name=None)

Update the widget when the iterated tag is modified.

Parameters:

tag_name – name of the iterated tag

update_selected_tag(selected_tag)

blabla

update_table()

Update the iteration table.

populse_mia.user_interface.pipeline_manager.node_controller module

Module to handle the node of a pipeline and its plugs.

Contains:

Class:

• PlugFilter (must be declared before AttributesFilter)

• AttributesFilter

• CapsulNodeController

• FilterWidget

• NodeController

class populse_mia.user_interface.pipeline_manager.node_controller.AttributesFilter(project, scans_list, process, node_name, plug_name, node_controller, main_window)

Bases: PlugFilter

Filter widget used on an attributes set for completion.

The widget displays a browser with the selected files of the database, a rapid search and an advanced search to filter these files. Once the filtering is done, the result (as a list of files) is set to the plug.

attributes_selected

ok_clicked()

Close the widget

class populse_mia.user_interface.pipeline_manager.node_controller.CapsulNodeController(project, scan_list, pipeline_manager_tab, main_window)

Bases: QWidget

Implementation of NodeController using Capsul AttributedProcessWidget widget.

__init__(project, scan_list, pipeline_manager_tab, main_window)

display_parameters(node_name, process, pipeline)

Display the parameters of the selected node.

The node parameters are read and line labels/line edits/push buttons are created for each of them. This methods consists mainly in widget and layout organization.

Parameters:

• node_name – name of the node

• process – process of the node

• pipeline – current pipeline

filter_attributes()

Display a filter widget.

parameters_changed(_, plug_name, old_value, new_value)

Emit the value_changed signal.

release_process()

Remove notification from process

rename_subprocesses(node, parent_node_name)

Change the name of a node.

static static_release(process, param_changed)

Remove notification

update_attributes_from_filter(attributes)

Update attributes from filter widget

update_node_name(new_node_name=None, old_node_name=None, from_undo=False, from_redo=False)

Change the name of the selected node and updates the pipeline.

Because the nodes are stored in a dictionary, we have to create a new node that has the same traits as the selected one and create new links that are the same than the selected node.

Parameters:

• new_node_name – new node name (is None except when this method is called from an undo/redo)

• old_node_name – old node name (is None except when this method is called from an undo/redo)

• from_undo – boolean, True if the action has been made using an undo

• from_redo – boolean, True if the action has been made using a redo

update_parameters(process=None)

Update the parameters values.

Does nothing any longer since the controller widget already reacts to changes in the process parameters.

Parameters:

process – process of the node

value_changed

class populse_mia.user_interface.pipeline_manager.node_controller.FilterWidget(project, node_name, node, main_window)

Bases: QWidget

Filter widget used on a Input_Filter process.

The widget displays a browser with the selected files of the database, a rapid search and an advanced search to filter these files. Once the filtering is done, the filter is saved in the process.

__init__(project, node_name, node, main_window)

Initialization of the Filter Widget.

Parameters:

• project – current project in the software

• node_name – name of the current node

• node – instance of the corresponding Input_Filter node

• main_window – parent main window

layout_view()

Create the layout.

ok_clicked()

Set the filter to the process and closes the widget.

reset_search_bar()

Reset the search bar of the rapid search.

search_str(str_search)

Update the files to display in the browser.

Parameters:

str_search – string typed in the rapid search

update_tag_to_filter()

Update the tag to Filter.

update_tags()

Update the list of visualized tags.

class populse_mia.user_interface.pipeline_manager.node_controller.NodeController(project, scan_list, pipeline_manager_tab, main_window)

Bases: QWidget

Allow to change the input and output values of a pipeline node

__init__(project, scan_list, pipeline_manager_tab, main_window)

Initialization of the Node Controller

Parameters:

• project – current project in the software

• scan_list – list of the selected database files

• pipeline_manager_tab – parent widget

• main_window – main window of the software

clearLayout(layout)

Clear the layouts of the widget.

Parameters:

layout – widget with a layout

display_filter(node_name, plug_name, parameters, process)

Display a filter widget.

Parameters:

• node_name – name of the node

• plug_name – name of the plug

• parameters – tuple containing the index of the plug, the current pipeline instance and the type of the plug value

• process – process of the node

display_parameters(node_name, process, pipeline)

Display the parameters of the selected node.

The node parameters are read and line labels/line edits/push buttons are created for each of them. This methods consists mainly in widget and layout organization.

Parameters:

• node_name – name of the node

• process – process of the node

• pipeline – current pipeline

get_index_from_plug_name(plug_name, in_or_out)

Return the index of the plug label.

Parameters:

• plug_name – name of the plug

• in_or_out – “in” if the plug is an input plug, “out” else

Returns:

the corresponding index

release_process()

Remove notification from process

rename_subprocesses(node, parent_node_name)

Change the name of a node.

update_node_name(new_node_name=None)

Change the name of the selected node and updates the pipeline.

Because the nodes are stored in a dictionary, we have to create a new node that has the same traits as the selected one and create new links that are the same than the selected node.

Parameters:

new_node_name – new node name (is None except when this method is called from an undo/redo)

update_parameters(process=None)

Update the parameters values.

Parameters:

process – process of the node

update_plug_value(in_or_out, plug_name, pipeline, value_type, new_value=None)

Update the value of a node plug.

Parameters:

• in_or_out – “in” if the plug is an input plug, “out” else

• plug_name – name of the plug

• pipeline – current pipeline

• value_type – type of the plug value

• new_value – new value for the plug (is None except when this method is called from an undo/redo)

update_plug_value_from_filter(plug_name, parameters, filter_res_list)

Update the plug value from a filter result.

Parameters:

• plug_name – name of the plug

• parameters – tuple containing the index of the plug, the current pipeline instance and the type of the plug value

• filter_res_list – list of the filtered files

value_changed

class populse_mia.user_interface.pipeline_manager.node_controller.PlugFilter(project, scans_list, process, node_name, plug_name, node_controller, main_window)

Bases: QWidget

Filter widget used on a node plug.

The widget displays a browser with the selected files of the database, a rapid search and an advanced search to filter these files. Once the filtering is done, the result (as a list of files) is set to the plug.

__init__(project, scans_list, process, node_name, plug_name, node_controller, main_window)

Initialization of the PlugFilter widget

Parameters:

• project – current project in the software

• scans_list – list of database files to filter

• process – process instance of the selected node

• node_name – name of the current node

• plug_name – name of the selected node plug

• node_controller – parent node controller

• main_window – parent main window

ok_clicked()

Set the new value to the node plug and closes the widget.

plug_value_changed

reset_search_bar()

Reset the search bar of the rapid search.

search_str(str_search)

Update the files to display in the browser. :param str_search: string typed in the rapid search

set_plug_value()

Emit a signal to set the file names to the node plug.

update_tag_to_filter()

Update the tag to Filter.

update_tags()

Update the list of visualized tags.

populse_mia.user_interface.pipeline_manager.node_controller.values(d)

blabla

populse_mia.user_interface.pipeline_manager.pipeline_editor module

Module that execute the pipeline manager tab menu actions and allow to edit graphically a pipeline.

Contains:

Class:

• PipelineEditor

• PipelineEditorTabs

Function:

• save_pipeline

• get_path

• find_filename

• values

class populse_mia.user_interface.pipeline_manager.pipeline_editor.PipelineEditor(project, main_window)

Bases: PipelineDeveloperView

View to edit a pipeline graphically.

__init__(project, main_window)

Initialization of the PipelineEditor.

Parameters:

• project – current project in the software

• main_window – current main window

add_link(source, dest, active, weak, from_undo=False, from_redo=False, allow_export=False)

Add a link between two nodes.

Parameters:

• source – tuple containing the node and plug source names

• dest – tuple containing the node and plug destination names

• active – boolean that is True if the link is activated

• weak – boolean that is True if the link is weak

• from_undo – boolean that is True if the action has been made using an undo

• from_redo – boolean that is True if the action has been made using a redo

add_named_process(class_process, node_name=None, from_undo=False, from_redo=False, links=[])

Add a process to the pipeline.

Parameters:

• class_process – process class’s name (str)

• node_name – name of the corresponding node (using when undo/redo) (str)

• from_undo – boolean that is True if the action has been made using an undo

• from_redo – boolean that is True if the action has been made using a redo

• links – list of links (using when undo/redo)

check_modifications()

Check if the nodes of the pipeline have been modified.

del_node(node_name=None, from_undo=False, from_redo=False)

Delete a node.

Parameters:

• node_name – name of the corresponding node (using when undo/redo)

• from_undo – boolean, True if the action has been made using an undo

• from_redo – boolean, True if the action has been made using a redo

export_node_plugs(node_name, inputs=True, outputs=True, optional=False, from_undo=False, from_redo=False)

Export all the plugs of a node

Parameters:

• node_name – node name

• inputs – True if the inputs have to be exported

• outputs – True if the outputs have to be exported

• optional – True if the optional plugs have to be exported

• from_undo – True if this method is called from an undo action

• from_redo – True if this method is called from a redo action

get_current_filename()

Return the relative path the pipeline was last saved to. Empty if never saved.

Returns:

the current pipeline file name

pipeline_modified

The signal that will be emitted when the pipeline is modified.

pipeline_saved

The signal that will be emitted when the pipeline is saved.

save_pipeline(filename=None)

Save the pipeline.

Returns:

the name of the file where the pipeline was saved

update_history(history_maker, from_undo, from_redo)

Update the history for undos and redos. This method is called after each action in the PipelineEditor.

Parameters:

• history_maker – list that contains information about what has been done

• from_undo – boolean that is True if the action has been made using an undo

• from_redo – boolean that is True if the action has been made using a redo

update_node_name(old_node, old_node_name, new_node_name, from_undo=False, from_redo=False)

Update a node name.

Parameters:

• old_node – Node object to change

• old_node_name – original name of the node (str)

• new_node_name – new name of the node (str)

• from_undo – boolean, True if the action has been made using an undo

• from_redo – boolean, True if the action has been made using a redo

update_plug_value(node_name, new_value, plug_name, value_type, from_undo=False, from_redo=False)

Update a plug value.

Parameters:

• node_name – name of the node (str)

• new_value – new value to set to the plug

• plug_name – name of the plug to change (str)

• value_type – type of the new value

• from_undo – boolean, True if the action has been made using an undo

• from_redo – boolean, True if the action has been made using a redo

class populse_mia.user_interface.pipeline_manager.pipeline_editor.PipelineEditorTabs(project, scan_list, main_window)

Bases: QTabWidget

Tab widget that contains pipeline editors.

__init__(project, scan_list, main_window)

Initialization of the Pipeline Editor tabs.

Parameters:

• project – current project in the software

• scan_list – list of the selected database files

• main_window – main window of the software

check_modifications(current_index)

Check if the nodes of the current pipeline have been modified.

Parameters:

current_index – index to check

close_tab(idx)

Close the selected tab and editor.

Parameters:

idx – index of the tab to close

emit_node_clicked(node_name, process)

Emit a signal when a node is clicked.

Parameters:

• node_name – node name

• process – process of the corresponding node

emit_pipeline_saved(filename)

Emit a signal when a pipeline is saved.

Parameters:

filename – file name of the pipeline

emit_switch_clicked(node_name, switch)

Emit a signal when a switch is clicked.

Parameters:

• node_name – node name

• switch – process of the corresponding node

export_to_db_scans(node_name)

Export the input of a filter to “database_scans” plug.

Parameters:

node_name – the name of the node from which to export

get_capsul_engine()

Get a CapsulEngine object from the edited pipeline, and set it up from MIA config object

get_current_editor()

Get the instance of the current editor.

Returns:

the current editor

get_current_filename()

Get the relative path to the file the pipeline in the current editor has been last saved to. If the pipeline has never been saved, returns the title of the tab.

Returns:

the filename of the current editor

get_current_pipeline()

Get the instance of the current pipeline.

Returns:

the pipeline of the current editor

get_current_tab_name()

Get the tab name of the editor in the current tab.

Trailing “*” and ampersand (”&”) characters are removed.

Returns:

the current tab name

get_editor_by_file_name(file_name)

Get the instance of an editor from its file name.

Parameters:

file_name – name of the file the pipeline was last saved to

Returns:

the editor corresponding to the file name

get_editor_by_index(idx)

Get the instance of an editor from its index in the editors.

Parameters:

idx – index of the editor

Returns:

the editor corresponding to the index

get_editor_by_tab_name(tab_name)

Get the instance of an editor from its tab name.

Parameters:

tab_name – name of the tab

Returns:

the editor corresponding to the tab name

get_filename_by_index(idx)

Get the relative path to the file the pipeline in the editor at the given index has been last saved to. If the pipeline has never been saved, returns the title of the tab.

Parameters:

idx – index of the editor

Returns:

the file name corresponding to the index

get_index_by_editor(editor)

Get the index of the editor corresponding to the given editor.

Parameters:

editor – searched pipeline editor

Returns:

the index corresponding to the editor

get_index_by_filename(filename)

Get the index of the first editor corresponding to the given pipeline filename.

Parameters:

filename – filename of the searched pipeline

Returns:

the index corresponding to the file name

get_index_by_tab_name(tab_name)

Get the index of the editor corresponding to the given tab name.

Parameters:

tab_name – name of the tab with the searched pipeline

Returns:

the index corresponding to the tab name

get_tab_name_by_index(idx)

Get the tab name of the editor at the given index.

Trailing “*” and ampersand (”&”) characters are removed.

Parameters:

idx – index of the editor

Returns:

the tab name corresponding to the index

has_pipeline_nodes()

Check if any of the pipelines in the editor tabs have pipeline nodes.

Returns:

True or False depending on if there are nodes in the editors

load_pipeline(filename=None)

Load a new pipeline.

Parameters:

filename – not None only when this method is called from “open_sub_pipeline”

load_pipeline_parameters()

Load parameters to the pipeline of the current editor

new_tab()

Create a new tab and a new editor and makes the new tab current.

node_clicked

open_filter(node_name)

Open a filter widget.

Parameters:

node_name – name of the corresponding node

open_sub_pipeline(sub_pipeline)

Open a sub-pipeline in a new tab.

Parameters:

sub_pipeline – the pipeline to open

pipeline_saved

process_clicked

reset_pipeline()

Reset the pipeline of the current editor.

save_pipeline(new_file_name=None)

Save the pipeline of the current editor.

save_pipeline_parameters()

Save the pipeline parameters of the current editor.

set_current_editor_by_editor(editor)

Set the current editor.

Parameters:

editor – editor in the tab that should be made current

set_current_editor_by_file_name(file_name)

Set the current editor from file name.

Parameters:

file_name – name of the file the pipeline was last saved to

set_current_editor_by_tab_name(tab_name)

Set the current editor from tab name.

Parameters:

tab_name – name of the tab

set_tab_index(index)

Set the current tab index and disable the run pipeline action.

Parameters:

index – index of the editor

switch_clicked

update_current_node(index)

Update the node parameters

Parameters:

index – index of the editor

update_history(editor)

Update undo/redo history of an editor.

Parameters:

editor – editor

update_iteration_checkbox()

blabla

update_pipeline_editors(editor)

Update editor.

Parameters:

editor – editor

update_scans_list()

Update the list of database scans in every editor.

populse_mia.user_interface.pipeline_manager.pipeline_editor.find_filename(paths_list, packages_list, file_name)

Find the corresponding file name in the paths list of process_config.yml.

Parameters:

• paths_list – list of all the paths contained in process_config.yml

• packages_list – packages path

• file_name – name of the sub-pipeline

Returns:

name of the corresponding file if it is found, else None

populse_mia.user_interface.pipeline_manager.pipeline_editor.get_path(name, dictionary, prev_paths=None, pckg=None)

Return the package path to the selected sub-pipeline.

Parameters:

• name – name of the sub-pipeline

• dictionary – package tree

• prev_paths – paths of the last call of this function

• pckg – package root

Returns:

the package path of the sub-pipeline if it is found, else None

populse_mia.user_interface.pipeline_manager.pipeline_editor.save_pipeline(pipeline, filename)

Save the pipeline either in XML or .py source file.

Parameters:

• pipeline – the pipeline to save

• filename – name of the file where to save the pipeline

populse_mia.user_interface.pipeline_manager.pipeline_editor.values(d)

Return a variable as a list.

Returns:

list

populse_mia.user_interface.pipeline_manager.pipeline_manager_tab module

Module to define pipeline manager tab appearance, settings and methods.

Contains:

Class:

• PipelineManagerTab

• RunProgress

• RunWorker

• StatusWidget

class populse_mia.user_interface.pipeline_manager.pipeline_manager_tab.PipelineManagerTab(project, scan_list, main_window)

Bases: QWidget

Widget that handles the Pipeline Manager tab.

__init__(project, scan_list, main_window)

Initialization of the Pipeline Manager tab

Parameters:

• project – current project in the software

• scan_list – list of the selected database files

• main_window – main window of the software

add_plug_value_to_database(p_value, brick_id, history_id, node_name, plug_name, full_name, job, trait, inputs, attributes)

Add the plug value to the database.

Parameters:

• p_value – plug value, a file name or a list of file names (any)

• brick_id – brick uuid in the database (str)

• history_id – history uuid in the database (str)

• node_name – name of the node (str)

• plug_name – name of the plug (str)

• full_name – full name of the node, including parent brick(s) (str). If there is no parent brick, full_name = node_name.

• job – job containing the plug (Job)

• trait – handler of the plug trait, or sub-trait if the plug is a list (Trait). It will be used to check the value type (file or not).

• inputs – input values for the process/node (dict)

• attributes – attributes set coming from Capsul completion engine to be set on all outputs of the node (dict)

ask_iterated_pipeline_plugs(pipeline)

Opens a dialog to ask for each pipeline plug, if an iteration should iterate over it, or if it should not be iterated, or if it should be connected to a database filter (input_filter node)

build_iterated_pipeline()

Build a new pipeline with an iteration node, iterating over the current pipeline

check_requirements(environment='global')

Return the configuration of a pipeline as required.

cleanup_older_init()

Remove non-existent entries from the databrowser.

complete_pipeline_parameters(pipeline=None)

Complete pipeline parameters using Capsul’s completion engine mechanism. This engine works using a set of attributes which can be retreived from the database.

controller_value_changed(signal_list)

Update history when a pipeline node is changed

Parameters:

signal_list – list of the needed parameters to update history. [“plug_value” or “node_name”, node_name, old_value, plug_name, plug_name_type, new_value]

displayNodeParameters(node_name, process)

Display the node controller when a node is clicked

Parameters:

• node_name – name of the node to display parameters

• process – process instance of the corresponding node

Returns:

finish_execution()

Callback called after a pipeline execution ends (in any way)

garbage_collect()

Index finished brick executions, Collect obsolete bricks and data and remove them from the database

This performs a posptocessing on current and older pipelines, and cleans up the database.

get_capsul_engine()

Get a CapsulEngine object from the edited pipeline, and set it up from MIA config object

get_missing_mandatory_parameters()

check on missing parameters for each job

get_pipeline_or_process(pipeline=None)

Get either the input pipeline (in the editor GUI), or its unique child process, if it only has one unconnected child. It allows to use a single process node from the GUI as a pipeline to iterate or run.

init_pipeline(pipeline=None, pipeline_name='')

Initialize the current pipeline of the pipeline editor

Parameters:

• pipeline – not None if this method call a sub-pipeline

• pipeline_name – name of the parent pipeline

initialize()

Clean previous initialization then initialize the current pipeline.

item_library_clicked

layout_view()

Initialize layout for the pipeline manager tab

loadParameters()

Load pipeline parameters to the current pipeline of the pipeline editor

loadPipeline()

Load a pipeline to the pipeline editor

postprocess_pipeline_execution(pipeline=None)

Operations to be performed after a run has been completed. It can be called either within the run procedure (the user clicks on the “run” button and waits for the results), or after a disconnetion / reconnection of the client app: the user clicks on “run” with distributed/remote execution activated, then closes the client MIA. Processing takes place (possibly remotely) within a soma-workflow server. Then the user runs MIA again, and we have to collect the outputs of runs which happenned (finished) while we were disconnected.

Such post-processing includes database indexing of output data, and should take into account not only the current pipeline, bu all past runs which have not been postprocessed yet.

When called with a pipeline argument, it only deals with this one.

The method can be called from within a worker run thread, thus has to be thread-safe.

Question: do we have to postprocess failed runs (pipelines which started and failed) ? Probably yes because they may have produced some results during the first steps, and failed later.

Question: how to decide which pipelines / runs have to be posptocessed now ? A pipeline may be started, then stopped or could have failed, then be postprocessed. But the user can still restart them in soma-workflow (or maybe mia one day), thus they should be postprocessed again then.

redo()

Redo the last undone action on the current pipeline editor

Actions that can be redone:

• add_process

• delete_process

• export_plug

• export_plugs

• remove_plug

• update_node_name

• update_plug_value

• add_link

• delete_link

register_completion_attributes(pipeline)

blabla

remove_progress()

blabla

runPipeline()

Run the current pipeline of the pipeline editor.

saveParameters()

Save the pipeline parameters of the the current pipeline of the pipeline editor

savePipeline(uncheck=False)

Save the current pipeline of the pipeline editor

Parameters:

uncheck – a flag to warn (False) or not (True) if a pipeline is going to be overwritten during saving operation

savePipelineAs()

Save the current pipeline of the pipeline editor under another name

show_status()

Show the last run status and execution info, errors etc.

stop_execution()

Request interruption of pipeline execution

undo()

Undo the last action made on the current pipeline editor

Actions that can be undone:

• add_process

• delete_process

• export_plug

• export_plugs

• remove_plug

• update_node_name

• update_plug_value

• add_link

• delete_link

updateProcessLibrary(filename)

Update the library of processes when a pipeline is saved

Parameters:

filename – file name of the pipeline that has been saved

static update_auto_inheritance(node, job=None)

Try (as best as possible) to assign output parameters to input ones, to get database tags for them.

When a node has only one input with a value (filename) in the database, then output filenames are considered to inherit from it. When several input parameters have values in the database, then if they are all equal, we can fallback to the first case. When values are different, and have different database tags, then the ambiguity remains, and we keep track of the several possible inputs which can provide tags for outputs.

The process attribute auto_inheritance_dict is filled with these values. It’s a dict with the shape:

{output_filename: <input_spec>}

output_filename is the relative filename in the database

<input_spec> can be:

• a string: filename

• a dict:

  {input_param: input_filename}
  

auto_inheritance_dict is built automatically, and is used as a fallback to ProcessMIA inheritance_dict, built “manually” (specialized for each process) in the ProcessMIA.list_outputs() when the latter does not exist, or does not specify what an output inherits from.

If ambiguities still subsist, the MIA infrastructure will ask the user how to solve them, which is not very convenient, and error-prone, thus should be avoided.

update_inheritance(job, node)

Update the inheritance dictionary

update_node_list(brick=None)

Update the list of nodes in workflow

update_project(project)

Update the project attribute of several objects

Parameters:

project – current project in the software

update_scans_list(iteration_list, all_iterations_list)

Update the user-selected list of scans

Parameters:

iteration_list – current list of scans in the iteration table

update_user_buttons_states(index=-1)

Update the visibility of initialize/run/save actions according to pipeline state

update_user_mode()

Update the visibility of widgets/actions depending of the chosen mode

class populse_mia.user_interface.pipeline_manager.pipeline_manager_tab.RunProgress(pipeline_manager, settings=None)

Bases: QWidget

Create the pipeline progress bar and launch the thread.

The progress bar is closed when the thread finishes.

Parameters:

• pipeline_manager – A PipelineManagerTab

• settings – dictionary of settings when the pipeline is iterated

__init__(pipeline_manager, settings=None)

cleanup()

blabla

end_progress()

blabla

start()

blabla

stop_execution()

blabla

class populse_mia.user_interface.pipeline_manager.pipeline_manager_tab.RunWorker(pipeline_manager)

Bases: QThread

Run the pipeline

__init__(pipeline_manager)

run()

blabla

class populse_mia.user_interface.pipeline_manager.pipeline_manager_tab.StatusWidget(pipeline_manager)

Bases: QWidget

Status widget: displays info about the current or last pipeline execution

__init__(pipeline_manager)

toggle_soma_workflow(checked)

blabla

populse_mia.user_interface.pipeline_manager.process_library module

Module that contains class and methods to process the different libraries of the project.

Contains:

Class:

• DictionaryTreeModel

• InstallProcesses

• Node

• PackageLibrary

• PackageLibraryDialog

• ProcessHelp

• ProcessLibrary

• ProcessLibraryWidget

Functions:

• import_file

• node_structure_from_dict

class populse_mia.user_interface.pipeline_manager.process_library.DictionaryTreeModel(root, parent=None)

Bases: QAbstractItemModel

Data model providing a tree of an arbitrary dictionary.

__init__(root, parent=None)

Initialization of the DictionaryTreeModel class.

columnCount(parent)

Number of columns is always 1.

data(index, role)

Return the data requested by the view.

flags(index)

Everything is enabled and selectable. Only the leaves can be dragged.

getNode(index)

Return a Node() from given index.

headerData(section, orientation, role)

Return the name of the requested column.

index(row, column, parent)

Return an index from given row, column and parent.

insertRows(position, rows, parent=<PyQt5.QtCore.QModelIndex object>)

Insert rows from starting position and number given by rows.

mimeData(idxs)

Used when the widget is dragged by the user.

Parameters:

idxs – mouse event

Returns:

QMimeData object

mimeTypes()

Return a constant.

parent(index)

Return the parent from given index.

Parameters:

index – index

removeRows(position, rows, parent=<PyQt5.QtCore.QModelIndex object>)

Remove the rows from position to position+rows.

Parameters:

• position – the position of the node

• rows – the row of the tnode

• parent – the parent node

rowCount(parent)

The number of rows is the number of children.

Parameters:

parent – the parent of the node

setData(index, value, role=2)

This method gets called when the user changes data.

Parameters:

• index – index

• value – value

• role – Qt role

Returns:

boolean

to_dict()

Return the root node as a dictionary.

Returns:

dictionary

class populse_mia.user_interface.pipeline_manager.process_library.InstallProcesses(main_window, folder)

Bases: QDialog

A widget that allows to browse a Python package or a zip file to install the processes that it is containing.

Parameters:

• main_window – current main window

• folder – boolean, True if folder, False if zip

__init__(main_window, folder)

Initialize the InstallProcesses class.

Parameters:

• main_window – current main window

• folder – boolean, True if folder, False if zip

get_filename(folder)

Open a file dialog to get the folder or zip file to install.

Parameters:

folder – True if the dialog installs from folder, False if from zip file

install()

Install a package from a zip file or a folder.

Contains: Private function:

• _add_package: Add a package and its modules to the package tree

• _change_pattern_in_folder: Changing all ‘old_pattern’ pattern to ‘new_pattern’ in the ‘path’ folder

process_installed

class populse_mia.user_interface.pipeline_manager.process_library.Node(name, parent=None)

Bases: object

Class to handle a package children.

__init__(name, parent=None)

Initialization of the Node class.

addChild(child)

Add a child to the children list.

Parameters:

child – child to add

attrs()

Returns:

child(row)

Return a child from its index in the list.

Parameters:

row – index in the list of children

Returns:

child

childCount()

Return the number of children.

Returns:

the length of the children list

data(column)

Return the name or the value of the object.

Parameters:

column – 0 for name, 1 for value

Returns:

string

insertChild(position, child)

Insert a child to a specific position.

Parameters:

• position – position

• child – child

Returns:

boolean, True if the insertion was successful

log(tabLevel=-1)

Return the logs.

Parameters:

tabLevel – Number of tabulation

Returns:

string

property name

Get the name of the object.

parent()

Return the parent of the object.

Returns:

parent

removeChild(position, child)

Remove a specific child.

Parameters:

• position – position of the child

• child – child to remove

Returns:

boolean, True if the child was removed

resource()

Return None

Returns:

None

row()

Return the index of the object in its parent list of children.

Returns:

index

setData(column, value)

Update the name or the value of the object.

Parameters:

• column – 0 for name, 1 for value

• value – new value

to_dict(d={})

Return a dictionary of the children.

Parameters:

d – dictionary

Returns:

dictionary of children

to_list()

Return the list of children with their names and values.

Returns:

list

property value

Get the value of the object.

class populse_mia.user_interface.pipeline_manager.process_library.PackageLibrary(package_tree, paths)

Bases: QTreeWidget

Tree that displays the user-added packages and their modules. The user can check or not each module/package.

__init__(package_tree, paths)

Initialization of the PackageLibrary widget.

Parameters:

• package_tree – representation of the packages as a tree-dictionary

• paths – list of paths to add to the system to import the packages

fill_item(item, value)

Fill the items of the tree recursively.

Parameters:

• item – current item to fill

• value – value of the item in the tree

generate_tree()

Generate the package tree

recursive_checks(parent)

Check/uncheck all child items.

Parameters:

parent – parent item

recursive_checks_from_child(child)

Check/uncheck all parent items.

Parameters:

child – child item

set_module_view(item, state)

Set if a module has to be enabled or disabled in the process library.

Parameters:

• item – item selected in the current tree

• state – checked or not checked (Qt.Checked == 2. So if val == 2 -> checkbox is checked, and if val == 0 -> checkbox is not checked)

update_checks(item, column)

Update the checks of the tree from an item.

Parameters:

• item – item on which to begin

• column – column from the check (should always be 0)

class populse_mia.user_interface.pipeline_manager.process_library.PackageLibraryDialog(mia_main_window=None, parent=None)

Bases: QDialog

Dialog that controls which processes to show in the process library.

__init__(mia_main_window=None, parent=None)

Initialization of the PackageLibraryDialog widget

add_package(module_name, class_name=None, show_error=False, init_package_tree=False)

Add a package and its modules to the package tree.

Parameters:

• module_name – name of the module

• class_name – name of the class

• show_error – display error in a message box in case of error. If False, errors are silent and error messages returned at the end of execution

• init_package_tree – boolean to initialize the package tree

add_package_with_text(_2add=False, update_view=True)

Add a package from the line edit’s text. :param _2add: name of package :param update_view: boolean to update the QListWidget

del_list

self.add_list.setSelectionMode(

QtGui.QAbstractItemView.ExtendedSelection)

self.remove_list.setSelectionMode(

QtGui.QAbstractItemView.ExtendedSelection)

self.del_list.setSelectionMode(

QtGui.QAbstractItemView.ExtendedSelection)

delete_package(index=1, to_delete=None, remove=True, loop=False, from_pipeline_manager=False)

Delete a package, only available to administrators.

Remove the package from the package library tree, update the __init__ file and delete the package directory and files if there are empty.

Parameters:

• index – recursive index to move between modules

• to_delete – the brick (class) do delete (ex. test.Test)

• remove – (boolean) if True, remove the brick(s) from the package tree

• loop – (boolean) to loop silently (if True no confirmation is requested before deletion)

Returns:

the list of deleted brick (class)

delete_package_with_text(_2del=False, update_view=True)

Delete a package from the line edit’s text. :param _2del: name of package :param update_view: boolean to update the QListWidget

install_processes_pop_up(folder=False)

Open the install processes pop-up.

Parameters:

folder – boolean, True if installing from a folder

static load_config()

Update the config and loads the corresponding packages.

Returns:

the config as a dictionary

load_packages()

Update the tree of the process library.

ok_clicked()

Called when apply changes is clicked.

remove_package(package)

Remove a package from the package tree.

Parameters:

package – module’s representation as a string (e.g.: nipype.interfaces.spm)

Returns:

True if the package has been removed correctly

remove_package_with_text(_2rem=None, update_view=True, tree_remove=True)

Remove the package in the line edit from the package tree.

Parameters:

• _2rem – name of package

• update_view – boolean to update the QListWidget

• tree_remove – boolean, remove from the tree

reset_action(itemlist, add)

Called to reset a prevous add or remove package action.

Parameters:

• itemlist – the QListWidget from add or remove package

• add – boolean to know which list to update

save(close=True)

Save the tree to the process_config.yml file.

save_config()

Save the current config to process_config.yml.

signal_save

update_config()

Update the process_config and package_library attributes.

class populse_mia.user_interface.pipeline_manager.process_library.ProcessHelp(process)

Bases: QWidget

A widget that displays information about the selected process.

Parameters:

process – selected process

__init__(process)

Generate the help.

Parameters:

process – selected process

class populse_mia.user_interface.pipeline_manager.process_library.ProcessLibrary(d, pkg_lib)

Bases: QTreeView

Tree to display the available Capsul’s processes

Parameters:

d – dictionary corresponding to the tree (dict)

__init__(d, pkg_lib)

Initialization of the ProcessLibrary class.

Parameters:

• d – dictionary: dictionary corresponding to the tree

• pkg_lib – an instance of the PackageLibraryDialog class

item_library_clicked

keyPressEvent(event)

Event when the delete key is pressed.

load_dictionary(d)

Load a dictionary to the tree.

Parameters:

d – dictionary to load. See the packages attribute in the ProcessLibraryWidget class

mousePressEvent(event)

Event when the mouse is pressed.

to_dict()

Return a dictionary from the current tree.

Returns:

the dictionary of the tree

class populse_mia.user_interface.pipeline_manager.process_library.ProcessLibraryWidget(main_window=None)

Bases: QWidget

Widget that handles the available Capsul’s processes in the software.

Parameters:

main_window – current main window

__init__(main_window=None)

Initialize the ProcessLibraryWidget.

Parameters:

main_window – current main window

static load_config()

Read the config in process_config.yml and return it as a dictionary.

Returns:

the config as a dictionary

load_packages()

Set packages and paths to the widget and to the system paths.

open_pkg_lib()

Open the package library.

save_config()

Save the current config to process_config.yml.

update_config()

Update the config and loads the corresponding packages.

update_process_library()

Update the tree of the process library.

populse_mia.user_interface.pipeline_manager.process_library.import_file(full_name, path)

Import a python module from a path (3.4+ only).

Does not call sys.modules[full_name] = path

Parameters:

• full_name – name of the package

• path – path of the package

Returns:

the corresponding module

populse_mia.user_interface.pipeline_manager.process_library.node_structure_from_dict(datadict, parent=None, root_node=None)

Return a hierarchical node stucture required by the TreeModel.

Parameters:

• datadict – dictionary

• parent – Parent of the node

• root_node – Root of the node

populse_mia.user_interface.pipeline_manager.process_mia module

Module used by mia_processes bricks to run processes.

Contains:

Class:

• MIAProcessCompletionEngine

• MIAProcessCompletionEngineFactory

• ProcessMIA

class populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngine(process, name, fallback_engine)

Bases: ProcessCompletionEngine

A specialized ProcessCompletionEngine for completion of all processes within the Populse_MIA context.

PopulseMIA processes instances and :class:NipypeProcess` instances have a special handling.

PopulseMIA processes use their method ProcessMIA.list_outputs() to perform completion from given input parameters. It is currently not based on attributes like in capsul completion, but on filenames.

Processes also get their matlab / SPM settings filled in from the config if they need them (NipypeProcess instances).

If the process use it and it is in the study config, their “project” parameter is also filled in, as well as the “output_directory” parameter.

The “normal” Capsul completion system is also complemented using MIA database: attributes from input parameters in the database (called “tags” here in MIA) are added to the completion attributes.

The MIA project will keep track of completed processes, in the correct completion order, so that other operations can be performed following the same order later after completion.

Contains:

Method:

• __init__: constructor

• complete_attributes_with_database: augments the Capsul completion system attributes associated with a process

• complete_nipype_common: set Nipype parameters for SPM

• complete_parameters: completes file parameters from given inputs parameters

• complete_parameters_mia: Completion for ProcessMIA instances

• get_attribute_values: re-route to underlying fallback engine

• get_path_completion_engine: re-route to underlying fallback engine

• get_project: get the project associated with the process

• path_attributes: re-route to underlying fallback engine

• remove_switch_observe: reimplemented since it is expectes in switches completion engine

__init__(process, name, fallback_engine)

Constructor.

Parameters

process: a process name: the name of the process fallback_engine: the fallback engine for the process

complete_attributes_with_database(process_inputs={})

Augments the Capsul completion system attributes associated with a process. Attributes from the database are queried for input parameters, and added to the completion attributes values, if they match.

Parameters

process_inputs: dict (optional)

parameters to be set on the process.

static complete_nipype_common(process, output_dir=True)

Set Nipype parameters for SPM. This is used both on NipypeProcess and ProcessMIA instances which have the appropriate parameters.

Parameters

process: a process output_dir: a boolean. If False, the output_directory attribute value is not initialised

complete_parameters(process_inputs={}, complete_iterations=True)

Completes file parameters from given inputs parameters.

Parameters

process_inputs: dict (optional)

parameters to be set on the process. It may include “regular” process parameters, and attributes used for completion. Attributes should be in a sub-dictionary under the key “capsul_attributes”.

complete_iterations: bool (optional)

if False, iteration nodes inside the pipeline will not run their own completion. Thus parameters for the iterations will not be correctly completed. However this has 2 advantages: 1. it prevents modification of the input pipeline, 2. it will not do iterations completion which will anyway be done (again) when building a workflow in ~capsul.pipeline.pipeline_workflow.workflow_from_pipeline.

complete_parameters_mia(process_inputs={})

Completion for ProcessMIA instances. This is done using their :meth: ProcessMIA.list_outputs method, which fills in output parameters from input values, and sets the internal inheritance_dict used after completion for data indexation in MIA.

Parameters

process_inputs: dict (optional)

parameters to be set on the process.

get_attribute_values()

Re-route to underlying fallback engine.

get_path_completion_engine()

Re-route to underlying fallback engine.

static get_project(process)

Get the project associated with the process.

Parameters

process: a process

Returns

project: the project associated with the process

path_attributes(filename, parameter=None)

Re-route to underlying fallback engine.

Parameters

filename: parameter:

Returns

out: the path attributes

remove_switch_observer(observer=None)

Reimplemented since it is expectes in switches completion engine.

Parameters

observer:

Returns

out:

class populse_mia.user_interface.pipeline_manager.process_mia.MIAProcessCompletionEngineFactory

Bases: ProcessCompletionEngineFactory

Completion engine factory specialization for Popules MIA context. Its factory_id is “mia_completion”.

This factory is activated in the StudyConfig instance by setting 2 parameters:

study_config.attributes_schema_paths = study_config.            attributes_schema_paths + ['populse_mia.user_interface.'
                               'pipeline_manager.process_mia']
study_config.process_completion =  'mia_completion'

Once this is done, the completion system will be activated for all processes, and use differently all MIA processes and nipype processes. For regular processes, additional database operations will be performed, then the underlying completion system will be called (FOM or other).

Contains:

Method:

• get_completion_engine: get a ProcessCompletionEngine instance for a given process/node

factory_id = 'mia_completion'

get_completion_engine(process, name=None)

Get a ProcessCompletionEngine instance for a given process/node.

Parameters

process: Node or Process instance name: str

Returns

out: ProcessCompletionEngine instance

class populse_mia.user_interface.pipeline_manager.process_mia.ProcessMIA(*args, **kwargs)

Bases: Process

Class overriding the default capsul Process class, in order to

customise the run for MIA bricks.

This class is mainly used by MIA bricks.

Note

• Type ‘ProcessMIA.help()’ for a full description of this process parameters.

• Type ‘<ProcessMIA>.get_input_spec()’ for a full description of this process input trait types.

• Type ‘<ProcessMIA>.get_output_spec()’ for a full description of this process output trait types.

__init__(*args, **kwargs)

Initialize the Process class.

ignore = {}

ignore_node = False

init_default_traits()

Automatically initialise necessary parameters for nipype or capsul

init_process(int_name)

Instantiation of the process attribute given an process identifier.

Parameters:

int_name – a process identifier

key = {}

list_outputs()

Override the outputs of the process.

make_initResult()

Make the initResult_dict from initialisation.

relax_nipype_exists_constraints()

Relax the exists constraint of the process.inputs traits

requirements()

Capsul Process.requirements() implementation using MIA’s ProcessMIA.requirement attribute

run_process_mia()

Implements specific runs for Process_Mia subclasses

tags_inheritance(in_file, out_file, node_name=None, own_tags=None, tags2del=None)

Create tags for data

Parameters:

• in_file – a process input file name (the document identifier for a database collection, a str, unambiguous case) from which the tags are inherited or a dictionary whose keys are a plug name and whose value is the corresponding process input file name (ambiguous case).

• out_file – a process output file name (the document_id for a database collection) which will inherit tags.

• node_name – the name of the node.

• own_tags – a list of dictionaries. Each dictionary corresponds to a tag to be added or modified. Mandatory keys (and associated values): “name”, “field_type”, “description”, “visibility”, “origin”, “unit”,”default_value”, “value”.

• tags2del – a list of tags (str) to delete (value)

populse_mia.user_interface.pipeline_manager.type_editors module

Define the Mia logger.

The soma control classes are overloaded for the needs of Mia.

Contains:

Class:

• PopulseFileControlWidget

• PopulseDirectoryControlWidget

• PopulseOffscreenListFileControlWidget

• PopulseUndefinedControlWidget

class populse_mia.user_interface.pipeline_manager.type_editors.PopulseDirectoryControlWidget

Bases: DirectoryControlWidget

Control to enter a Directory.

Contains:

Method:

• create_widget: method to create the file widget

• filter_clicked: display a filter widget

• update_plug_value_from_filter: update the plug value from a filter result

static create_widget(parent, control_name, control_value, trait, label_class=None, user_data=None)

Method to create the directory widget.

static filter_clicked(widget, node_name, plug_name)

Display a filter widget.

Parameters:

• node_name – name of the node

• plug_name – name of the plug

static update_plug_value_from_filter(widget, plug_name, filter_res_list)

Update the plug value from a filter result.

Parameters:

• plug_name – name of the plug

• filter_res_list – list of the filtered files

class populse_mia.user_interface.pipeline_manager.type_editors.PopulseFileControlWidget

Bases: FileControlWidget

Control to enter a file.

Contains:

Method:

• create_widget: method to create the file widget

• filter_clicked: display a filter widget

• update_plug_value_from_filter: update the plug value from a filter result

static create_widget(parent, control_name, control_value, trait, label_class=None, user_data=None)

Method to create the file widget.

Parameters

parent: QWidget (mandatory)

the parent widget

control_name: str (mandatory)

the name of the control we want to create

control_value: str (mandatory)

the default control value

trait: Tait (mandatory)

the trait associated to the control

label_class: Qt widget class (optional, default: None)

the label widget will be an instance of this class. Its constructor will be called using 2 arguments: the label string and the parent widget.

Returns

out: 2-uplet

a two element tuple of the form (control widget: QWidget with two elements, a QLineEdit in the ‘path’ parameter and a browse button in the ‘browse’ parameter, associated label: QLabel)

static filter_clicked(widget, node_name, plug_name)

Display a filter widget.

Parameters:

• node_name – name of the node

• plug_name – name of the plug

static update_plug_value_from_filter(widget, plug_name, filter_res_list)

Update the plug value from a filter result.

Parameters:

• plug_name – name of the plug

• filter_res_list – list of the filtered files

class populse_mia.user_interface.pipeline_manager.type_editors.PopulseOffscreenListFileControlWidget

Bases: OffscreenListFileControlWidget

Control to enter a list of files.

Contains:

Method:

• create_widget: method to create the list of files widget

• filter_clicked: display a filter widget

• update_plug_value_from_filter: update the plug value from a filter result

static create_widget(parent, control_name, control_value, trait, label_class=None, user_data=None)

Method to create the list of files widget.

Parameters

parent: QWidget (mandatory)

the parent widget

control_name: str (mandatory)

the name of the control we want to create

control_value: list of items (mandatory)

the default control value

trait: Tait (mandatory)

the trait associated to the control

label_class: Qt widget class (optional, default: None)

the label widget will be an instance of this class. Its constructor will be called using 2 arguments: the label string and the parent widget.

Returns

out: 2-uplet

a two element tuple of the form (control widget: , associated labels: (a label QLabel, the tools QWidget))

static filter_clicked(widget, node_name, plug_name)

Display a filter widget.

Parameters:

• node_name – name of the node

• plug_name – name of the plug

static update_plug_value_from_filter(widget, plug_name, filter_res_list)

Update the plug value from a filter result.

Parameters:

• plug_name – name of the plug

• filter_res_list – list of the filtered files

class populse_mia.user_interface.pipeline_manager.type_editors.PopulseUndefinedControlWidget

Bases: object

Control for Undefined value.

classmethod check(control_instance)

Check if a controller widget control is filled correctly.

Parameters

cls: StrControlWidget (mandatory)

a StrControlWidget control

control_instance: QLineEdit (mandatory)

the control widget we want to validate

classmethod connect(controller_widget, control_name, control_instance)

Connect a ‘Str’ or ‘String’ controller trait and a ‘StrControlWidget’ controller widget control.

Parameters

cls: StrControlWidget (mandatory)

a StrControlWidget control

controller_widget: ControllerWidget (mandatory)

a controller widget that contains the controller we want to update

control_name: str (mandatory)

the name of the controller widget control we want to synchronize with the controller

control_instance: QLineEdit (mandatory)

the instance of the controller widget control we want to synchronize with the controller

static create_widget(parent, control_name, control_value, trait, label_class=None, user_data=None)

Method to create the Undefined widget.

Parameters

parent: QWidget (mandatory)

the parent widget

control_name: str (mandatory)

the name of the control we want to create

control_value: str (mandatory)

the default control value

trait: Tait (mandatory)

the trait associated to the control

label_class: Qt widget class (optional, default: None)

the label widget will be an instance of this class. Its constructor will be called using 2 arguments: the label string and the parent widget.

Returns

out: 2-uplet

a two element tuple of the form (control widget: QLineEdit, associated label: QLabel)

static disconnect(controller_widget, control_name, control_instance)

Disconnect a ‘Str’ or ‘String’ controller trait and a ‘StrControlWidget’ controller widget control.

Parameters

controller_widget: ControllerWidget (mandatory)

a controller widget that contains the controller we want to update

control_name: str(mandatory)

the name of the controller widget control we want to synchronize with the controller

control_instance: QLineEdit (mandatory)

the instance of the controller widget control we want to synchronize with the controller

static is_valid(control_instance, *args, **kwargs)

Method to check if the new control value is correct.

Parameters

control_instance: QWidget (mandatory)

the control widget we want to validate

Returns

out: bool

True if the control value is Undefined, False otherwise

static update_controller(controller_widget, control_name, control_instance, reset_invalid_value, *args, **kwargs)

Update one element of the controller.

At the end the controller trait value with the name ‘control_name’ will match the controller widget user parameters defined in ‘control_instance’.

Parameters

controller_widget: ControllerWidget (mandatory)

a controller widget that contains the controller we want to update

control_name: str(mandatory)

the name of the controller widget control we want to synchronize with the controller

control_instance: QLineEdit (mandatory)

the instance of the controller widget control we want to synchronize with the controller

static update_controller_widget(controller_widget, control_name, control_instance)

Update one element of the controller widget.

At the end the controller widget user editable parameter with the name ‘control_name’ will match the controller trait value with the same name.

Parameters

controller_widget: ControllerWidget (mandatory)

a controller widget that contains the controller we want to update

control_name: str(mandatory)

the name of the controller widget control we want to synchronize with the controller

control_instance: QLineEdit (mandatory)

the instance of the controller widget control we want to synchronize with the controller