Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revision Previous revision
Next revision
Previous revision
auto3dem:userguide [2015/12/22 15:10]
gcardone [p3dr]
auto3dem:userguide [2016/01/19 13:04] (current)
admin [auto]
Line 192: Line 192:
 ppft bin_factor ​  2 # Start with 2x2 binning of images ppft bin_factor ​  2 # Start with 2x2 binning of images
 </​code>​ </​code>​
 +
 +==== Controlling distribution on CPUs in parallel runs ====
 +
 +All the programs capable of running on multiple processors use the Message Passing Interface (MPI) library for parallelization. In these case the processors are assigned automatically at launch by means of a machinefile,​ a file listing all the nodes assigned to the run. In Auto3DEM it is possible to control how these nodes are assigned and their scheduling method.
 +One control parameter is
 +
 +  auto evod_parallel 0/1
 +
 +If set to 1, orientation search and reconstructions from even and odd stacks are done at the same time (assuming that the program is running in gold standard mode). Specifically,​ the processors are divided into two halves, by affinity (i.e. as many processors in the same nodes), and two instances of the programs are launched concurrently.
 +
 +If the size of the reconstructions is large, compared to the memory available in a single node, then there is a risk that the reconstruction routine could freeze because of insufficient memory, especially when multiple processes are running on one node, therefore sharing the same memory. In this case it could be advantageous to spread the CPUs among separate nodes, so that they don't compete with each other for resources. This can be obtained by setting the parameter
 +
 +  p3dr cpu_sparse
 +
 +to 1, which distributes the processors among the two concurrent reconstructions according to a round-robin scheduling approach. For example, if you are launching auto3dem on 8 nodes, asking all the processors on each of them, while you limit P3DR to use only 4 cpus (''​p3dr max_cpu 4''​),​ these 4 cpus will be on separate nodes, and the two reconstructions from even- and odd-numbered particles will run at the same time, but each MPI process will be on a different node. Of course, this is kind of best case scenario.
  
 =====Full listing of auto3dem keywords===== =====Full listing of auto3dem keywords=====
Line 221: Line 236:
 **freeze_res**:​ non-zero value freezes the resolutions used in PPFT, and PO2R (resolution used in P3DR will still be based on the results from PSF). **freeze_res**:​ non-zero value freezes the resolutions used in PPFT, and PO2R (resolution used in P3DR will still be based on the results from PSF).
  
-**fsc_hithresh**: cutoff value for FSC used in estimating map resolution.+**fsc_cutoff**: cutoff value for FSC used in estimating map resolution. If not specified it is set either to 0.143 (gold_standard set to 1) or to 0.5 (gold_standard set to 0).
  
-**fsc_lothresh**: cutoff value for FSC used to set resolution limits in P3DR, PSF, and PO2R.+**fsc_hithresh**: cutoff value for FSC used to set resolution limits in PPFT and PO2R
 + 
 +**fsc_lothresh**:​ cutoff value for FSC used in filtering the map in P3DR.
  
 **gauss_adj**:​ parameter used to set width of Gaussian falloff in P3DR (reciprocal angstroms). **gauss_adj**:​ parameter used to set width of Gaussian falloff in P3DR (reciprocal angstroms).
Line 249: Line 266:
 **iter_start**:​ starting iteration, i.e. number assigned to first iteration. **iter_start**:​ starting iteration, i.e. number assigned to first iteration.
  
-**map_suffix**:​ string suffix to append to the name of the reconstruction. Used only if generate_map is set to ‘only’. Default is ‘none’, a reserved string indicating that the name is assigned according to the standard rules. :​!:​__Warning__‘none’ implies that for multiple runs of only-reconstructions each one will overwrite the previous result.+**map_suffix**:​ string suffix to append to the name of the reconstruction. Used only if generate_map is set to ‘only’. Default is ‘none’, a reserved string indicating that the name is assigned according to the standard rules. :!: __Warning__ ‘none’ implies that for multiple runs of only-reconstructions each one will overwrite the previous result.
  
 **mode**: AUTO3DEM mode of operation. Allowed values = (search, refine). **mode**: AUTO3DEM mode of operation. Allowed values = (search, refine).
Line 430: Line 447:
 **magref_reset**:​ 1 to ‘forget’ magnification assigned to  data, 0 to refine around the current estimated value. **magref_reset**:​ 1 to ‘forget’ magnification assigned to  data, 0 to refine around the current estimated value.
  
-**magref_step**:​ size of magnification refinement step (microns).+**magref_step**:​ size of magnification refinement step (fraction)
 + 
 +**mask_map**:​ name of binary map to use for focused refinement. Only used for modes //local// and //​ticos_equiv//​. Default is ‘none’, a reserved string indicating that no mask is used.
  
 **max_cpu**:​ maximum number of CPUs to be used by PO2R **max_cpu**:​ maximum number of CPUs to be used by PO2R
Line 437: Line 456:
 <WRAP center round box 75%> <WRAP center round box 75%>
 ''​local'':​ local refinement (//nangle// steps of //dangle// degrees along each direction)\\ ''​local'':​ local refinement (//nangle// steps of //dangle// degrees along each direction)\\
-''​mag'':​ magnification refinement (//​nmagref//​ steps of //​magref//​_step ​microns ​along each direction)\\+''​mag'':​ magnification refinement (//​nmagref//​ steps of //​magref//​_step ​fraction ​along each direction)\\
 ''​global'':​ global search in one asymmetric unit (on a grid with step of //gangle// degrees)\\ ''​global'':​ global search in one asymmetric unit (on a grid with step of //gangle// degrees)\\
 ''​ticos_equiv'':​ restricted search to the 60 symmetry related orientations. ''​ticos_equiv'':​ restricted search to the 60 symmetry related orientations.
Line 446: Line 465:
 **ncenter**:​ number of spatial steps taken in each direction. **ncenter**:​ number of spatial steps taken in each direction.
  
-**nmagf**: number of magnification factor steps along each direction.+**nmagref**: number of magnification factor steps along each direction.
  
 **per_ptle_ctf**:​ apply CTF correction on a per-particle basis. **per_ptle_ctf**:​ apply CTF correction on a per-particle basis.
Line 547: Line 566:
 | auto | delete_maps | 1 | | auto | delete_maps | 1 |
 | auto | estimate_res | 1 | | auto | estimate_res | 1 |
 +| auto | evod_parallel | 1 |
 | auto | flatten_falloff | -1 (automatic) | | auto | flatten_falloff | -1 (automatic) |
 | auto | flatten_map | 0 | | auto | flatten_map | 0 |
 | auto | freeze_annulus | 0 | | auto | freeze_annulus | 0 |
 | auto | freeze_res | 0 | | auto | freeze_res | 0 |
 +| auto | fsc_cutoff | 0.143 if gold_standard=1,​ 0.5 otherwise |
 | auto | fsc_hithresh | 0.5 | | auto | fsc_hithresh | 0.5 |
 | auto | fsc_lothresh | 0.3 | | auto | fsc_lothresh | 0.3 |
Line 612: Line 633:
 | p3dr | apo_border | 12 | | p3dr | apo_border | 12 |
 | p3dr | bin | P3DR | | p3dr | bin | P3DR |
 +| p3dr | cpu_sparse | 0 |
 | p3dr | ctf_ff1 | 1 | | p3dr | ctf_ff1 | 1 |
 | p3dr | ctf_ff2 | 0.1 | | p3dr | ctf_ff2 | 0.1 |
Line 657: Line 679:
 | po2r | magref_reset | 1 | | po2r | magref_reset | 1 |
 | po2r | magref_step | 0.005 | | po2r | magref_step | 0.005 |
 +| po2r | mask_map | none (reserved string) |
 | po2r | max_cpu | 256 | | po2r | max_cpu | 256 |
 | po2r | nangle | 4 | | po2r | nangle | 4 |