Skip to content

Commit

Permalink
Merge pull request yt-project#1372 from ngoldbaum/profile-stddev
Browse files Browse the repository at this point in the history
Deprecate profile.variance in favor of profile.standard_deviation
  • Loading branch information
Nathan Goldbaum authored May 8, 2017
2 parents 6c3b530 + 1a131e0 commit 6a7fa66
Show file tree
Hide file tree
Showing 5 changed files with 49 additions and 37 deletions.
21 changes: 11 additions & 10 deletions doc/source/analyzing/generating_processed_data.rst
Original file line number Diff line number Diff line change
Expand Up @@ -93,10 +93,10 @@ can be used to generate 1, 2, and 3D profiles.

Profile objects can be created from any data object (see :ref:`data-objects`,
specifically the section :ref:`available-objects` for more information) and are
best thought of as distribution calculations. They can either sum up or
average one quantity with respect to one or more other quantities, and they do
this over all the data contained in their source object. When calculating average
values, the variance will also be calculated.
best thought of as distribution calculations. They can either sum up or average
one quantity with respect to one or more other quantities, and they do this over
all the data contained in their source object. When calculating average values,
the standard deviation will also be calculated.

To generate a profile, one need only specify the binning fields and the field
to be profiled. The binning fields are given together in a list. The
Expand Down Expand Up @@ -131,16 +131,17 @@ have data.
print(profile.used)
If a weight field was given, the profile data will represent the weighted mean of
a field. In this case, the weighted variance will be calculated automatically and
can be access via the ``profile.variance`` attribute.
If a weight field was given, the profile data will represent the weighted mean
of a field. In this case, the weighted standard deviation will be calculated
automatically and can be access via the ``profile.standard_deviation``
attribute.

.. code-block:: python
print(profile.variance["gas", "temperature"])
print(profile.standard_deviation["gas", "temperature"])
A two-dimensional profile of the total gas mass in bins of density and temperature
can be created as follows:
A two-dimensional profile of the total gas mass in bins of density and
temperature can be created as follows:

.. code-block:: python
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -19,14 +19,14 @@
weight_field='cell_mass')

# Create arrays to plot.
radius = prof.x.value
mean = prof['gas', 'velocity_magnitude'].value
variance = prof.variance['gas', 'velocity_magnitude'].value
radius = prof.x
mean = prof['gas', 'velocity_magnitude']
std = prof.standard_deviation['gas', 'velocity_magnitude']

# Plot the average velocity magnitude.
plt.loglog(radius, mean, label='Mean')
# Plot the variance of the velocity magnitude.
plt.loglog(radius, variance, label='Standard Deviation')
plt.loglog(radius, std, label='Standard Deviation')
plt.xlabel('r [kpc]')
plt.ylabel('v [cm/s]')
plt.legend()
Expand Down
16 changes: 8 additions & 8 deletions doc/source/cookbook/simple_plots.rst
Original file line number Diff line number Diff line change
Expand Up @@ -80,17 +80,17 @@ See :ref:`how-to-make-1d-profiles` for more information.

.. yt_cookbook:: time_series_profiles.py

.. _cookbook-profile-variance:
.. _cookbook-profile-stddev:

Profiles with Variance Values
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Profiles with Standard Deviation
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This shows how to plot the variance for a 1D profile. In this example, we
manually create a 1D profile object, which gives us access to the variance
data.
See :ref:`how-to-make-1d-profiles` for more information.
This shows how to plot a 1D profile with error bars indicating the standard
deviation of the field values in each profile bin. In this example, we manually
create a 1D profile object, which gives us access to the standard deviation
data. See :ref:`how-to-make-1d-profiles` for more information.

.. yt_cookbook:: profile_with_variance.py
.. yt_cookbook:: profile_with_standard_deviation.py

Making Plots of Multiple Fields Simultaneously
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down
6 changes: 3 additions & 3 deletions yt/analysis_modules/halo_analysis/halo_callbacks.py
Original file line number Diff line number Diff line change
Expand Up @@ -246,9 +246,9 @@ def profile(halo, bin_fields, profile_fields, n_bins=32, extrema=None, logs=None
setattr(halo, storage, halo_store)
halo_store.update(prof_store)

if hasattr(my_profile, "variance"):
variance_store = dict([(field, my_profile.variance[field]) \
for field in my_profile.variance])
if my_profile.standard_deviation is not None:
variance_store = dict([(field, my_profile.standard_deviation[field]) \
for field in my_profile.standard_deviation])
variance_storage = "%s_variance" % storage
if hasattr(halo, variance_storage):
halo_variance_store = getattr(halo, variance_storage)
Expand Down
35 changes: 23 additions & 12 deletions yt/data_objects/profiles.py
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,8 @@
from yt.funcs import \
get_output_filename, \
ensure_list, \
iterable
iterable, \
issue_deprecation_warning
from yt.units.yt_array import \
array_like_field, \
YTQuantity
Expand Down Expand Up @@ -91,12 +92,22 @@ def __init__(self, data_source, weight_field = None):
self.field_info = {}
self.field_data = YTFieldData()
if weight_field is not None:
self.variance = YTFieldData()
self.standard_deviation = YTFieldData()
weight_field = self.data_source._determine_fields(weight_field)[0]
else:
self.standard_deviation = None
self.weight_field = weight_field
self.field_units = {}
ParallelAnalysisInterface.__init__(self, comm=data_source.comm)

@property
def variance(self):
issue_deprecation_warning("""
profile.variance incorrectly returns the profile standard deviation and has
been deprecated, use profile.standard_deviation instead."""
)
return self.standard_deviation

def add_fields(self, fields):
"""Add fields to profile
Expand Down Expand Up @@ -153,13 +164,13 @@ def _finalize_storage(self, fields, temp_storage):

all_val = np.zeros_like(temp_storage.values)
all_mean = np.zeros_like(temp_storage.mvalues)
all_var = np.zeros_like(temp_storage.qvalues)
all_std = np.zeros_like(temp_storage.qvalues)
all_weight = np.zeros_like(temp_storage.weight_values)
all_used = np.zeros_like(temp_storage.used, dtype="bool")

# Combine the weighted mean and variance from each processor.
# For two samples with total weight, mean, and variance
# given by w, m, and s, their combined mean and variance are:
# Combine the weighted mean and standard deviation from each processor.
# For two samples with total weight, mean, and standard deviation
# given by w, m, and s, their combined mean and standard deviation are:
# m12 = (m1 * w1 + m2 * w2) / (w1 + w2)
# s12 = (m1 * (s1**2 + (m1 - m12)**2) +
# m2 * (s2**2 + (m2 - m12)**2)) / (w1 + w2)
Expand All @@ -180,16 +191,16 @@ def _finalize_storage(self, fields, temp_storage):
all_store[p].weight_values)[all_store[p].used] / \
all_weight[all_store[p].used]

all_var[..., i][all_store[p].used] = \
(old_weight * (all_var[..., i] +
all_std[..., i][all_store[p].used] = \
(old_weight * (all_std[..., i] +
(old_mean[..., i] - all_mean[..., i])**2) +
all_store[p].weight_values *
(all_store[p].qvalues[..., i] +
(all_store[p].mvalues[..., i] -
all_mean[..., i])**2))[all_store[p].used] / \
all_weight[all_store[p].used]

all_var = np.sqrt(all_var)
all_std = np.sqrt(all_std)
del all_store
self.used = all_used
blank = ~all_used
Expand All @@ -206,10 +217,10 @@ def _finalize_storage(self, fields, temp_storage):
self.field_data[field] = \
array_like_field(self.data_source,
all_mean[...,i], field)
self.variance[field] = \
self.standard_deviation[field] = \
array_like_field(self.data_source,
all_var[...,i], field)
self.variance[field][blank] = 0.0
all_std[...,i], field)
self.standard_deviation[field][blank] = 0.0
self.field_data[field][blank] = 0.0
self.field_units[field] = self.field_data[field].units
if isinstance(field, tuple):
Expand Down

0 comments on commit 6a7fa66

Please sign in to comment.