comparison DEPENDENCIES/mingw32/Python27/Lib/site-packages/numpy/doc/indexing.py @ 87:2a2c65a20a8b

Add Python libs and headers
author Chris Cannam
date Wed, 25 Feb 2015 14:05:22 +0000
parents
children
comparison
equal deleted inserted replaced
86:413a9d26189e 87:2a2c65a20a8b
1 """
2 ==============
3 Array indexing
4 ==============
5
6 Array indexing refers to any use of the square brackets ([]) to index
7 array values. There are many options to indexing, which give numpy
8 indexing great power, but with power comes some complexity and the
9 potential for confusion. This section is just an overview of the
10 various options and issues related to indexing. Aside from single
11 element indexing, the details on most of these options are to be
12 found in related sections.
13
14 Assignment vs referencing
15 =========================
16
17 Most of the following examples show the use of indexing when
18 referencing data in an array. The examples work just as well
19 when assigning to an array. See the section at the end for
20 specific examples and explanations on how assignments work.
21
22 Single element indexing
23 =======================
24
25 Single element indexing for a 1-D array is what one expects. It work
26 exactly like that for other standard Python sequences. It is 0-based,
27 and accepts negative indices for indexing from the end of the array. ::
28
29 >>> x = np.arange(10)
30 >>> x[2]
31 2
32 >>> x[-2]
33 8
34
35 Unlike lists and tuples, numpy arrays support multidimensional indexing
36 for multidimensional arrays. That means that it is not necessary to
37 separate each dimension's index into its own set of square brackets. ::
38
39 >>> x.shape = (2,5) # now x is 2-dimensional
40 >>> x[1,3]
41 8
42 >>> x[1,-1]
43 9
44
45 Note that if one indexes a multidimensional array with fewer indices
46 than dimensions, one gets a subdimensional array. For example: ::
47
48 >>> x[0]
49 array([0, 1, 2, 3, 4])
50
51 That is, each index specified selects the array corresponding to the
52 rest of the dimensions selected. In the above example, choosing 0
53 means that remaining dimension of lenth 5 is being left unspecified,
54 and that what is returned is an array of that dimensionality and size.
55 It must be noted that the returned array is not a copy of the original,
56 but points to the same values in memory as does the original array.
57 In this case, the 1-D array at the first position (0) is returned.
58 So using a single index on the returned array, results in a single
59 element being returned. That is: ::
60
61 >>> x[0][2]
62 2
63
64 So note that ``x[0,2] = x[0][2]`` though the second case is more
65 inefficient a new temporary array is created after the first index
66 that is subsequently indexed by 2.
67
68 Note to those used to IDL or Fortran memory order as it relates to
69 indexing. Numpy uses C-order indexing. That means that the last
70 index usually represents the most rapidly changing memory location,
71 unlike Fortran or IDL, where the first index represents the most
72 rapidly changing location in memory. This difference represents a
73 great potential for confusion.
74
75 Other indexing options
76 ======================
77
78 It is possible to slice and stride arrays to extract arrays of the
79 same number of dimensions, but of different sizes than the original.
80 The slicing and striding works exactly the same way it does for lists
81 and tuples except that they can be applied to multiple dimensions as
82 well. A few examples illustrates best: ::
83
84 >>> x = np.arange(10)
85 >>> x[2:5]
86 array([2, 3, 4])
87 >>> x[:-7]
88 array([0, 1, 2])
89 >>> x[1:7:2]
90 array([1, 3, 5])
91 >>> y = np.arange(35).reshape(5,7)
92 >>> y[1:5:2,::3]
93 array([[ 7, 10, 13],
94 [21, 24, 27]])
95
96 Note that slices of arrays do not copy the internal array data but
97 also produce new views of the original data.
98
99 It is possible to index arrays with other arrays for the purposes of
100 selecting lists of values out of arrays into new arrays. There are
101 two different ways of accomplishing this. One uses one or more arrays
102 of index values. The other involves giving a boolean array of the proper
103 shape to indicate the values to be selected. Index arrays are a very
104 powerful tool that allow one to avoid looping over individual elements in
105 arrays and thus greatly improve performance.
106
107 It is possible to use special features to effectively increase the
108 number of dimensions in an array through indexing so the resulting
109 array aquires the shape needed for use in an expression or with a
110 specific function.
111
112 Index arrays
113 ============
114
115 Numpy arrays may be indexed with other arrays (or any other sequence-
116 like object that can be converted to an array, such as lists, with the
117 exception of tuples; see the end of this document for why this is). The
118 use of index arrays ranges from simple, straightforward cases to
119 complex, hard-to-understand cases. For all cases of index arrays, what
120 is returned is a copy of the original data, not a view as one gets for
121 slices.
122
123 Index arrays must be of integer type. Each value in the array indicates
124 which value in the array to use in place of the index. To illustrate: ::
125
126 >>> x = np.arange(10,1,-1)
127 >>> x
128 array([10, 9, 8, 7, 6, 5, 4, 3, 2])
129 >>> x[np.array([3, 3, 1, 8])]
130 array([7, 7, 9, 2])
131
132
133 The index array consisting of the values 3, 3, 1 and 8 correspondingly
134 create an array of length 4 (same as the index array) where each index
135 is replaced by the value the index array has in the array being indexed.
136
137 Negative values are permitted and work as they do with single indices
138 or slices: ::
139
140 >>> x[np.array([3,3,-3,8])]
141 array([7, 7, 4, 2])
142
143 It is an error to have index values out of bounds: ::
144
145 >>> x[np.array([3, 3, 20, 8])]
146 <type 'exceptions.IndexError'>: index 20 out of bounds 0<=index<9
147
148 Generally speaking, what is returned when index arrays are used is
149 an array with the same shape as the index array, but with the type
150 and values of the array being indexed. As an example, we can use a
151 multidimensional index array instead: ::
152
153 >>> x[np.array([[1,1],[2,3]])]
154 array([[9, 9],
155 [8, 7]])
156
157 Indexing Multi-dimensional arrays
158 =================================
159
160 Things become more complex when multidimensional arrays are indexed,
161 particularly with multidimensional index arrays. These tend to be
162 more unusal uses, but theyare permitted, and they are useful for some
163 problems. We'll start with thesimplest multidimensional case (using
164 the array y from the previous examples): ::
165
166 >>> y[np.array([0,2,4]), np.array([0,1,2])]
167 array([ 0, 15, 30])
168
169 In this case, if the index arrays have a matching shape, and there is
170 an index array for each dimension of the array being indexed, the
171 resultant array has the same shape as the index arrays, and the values
172 correspond to the index set for each position in the index arrays. In
173 this example, the first index value is 0 for both index arrays, and
174 thus the first value of the resultant array is y[0,0]. The next value
175 is y[2,1], and the last is y[4,2].
176
177 If the index arrays do not have the same shape, there is an attempt to
178 broadcast them to the same shape. If they cannot be broadcast to the
179 same shape, an exception is raised: ::
180
181 >>> y[np.array([0,2,4]), np.array([0,1])]
182 <type 'exceptions.ValueError'>: shape mismatch: objects cannot be
183 broadcast to a single shape
184
185 The broadcasting mechanism permits index arrays to be combined with
186 scalars for other indices. The effect is that the scalar value is used
187 for all the corresponding values of the index arrays: ::
188
189 >>> y[np.array([0,2,4]), 1]
190 array([ 1, 15, 29])
191
192 Jumping to the next level of complexity, it is possible to only
193 partially index an array with index arrays. It takes a bit of thought
194 to understand what happens in such cases. For example if we just use
195 one index array with y: ::
196
197 >>> y[np.array([0,2,4])]
198 array([[ 0, 1, 2, 3, 4, 5, 6],
199 [14, 15, 16, 17, 18, 19, 20],
200 [28, 29, 30, 31, 32, 33, 34]])
201
202 What results is the construction of a new array where each value of
203 the index array selects one row from the array being indexed and the
204 resultant array has the resulting shape (size of row, number index
205 elements).
206
207 An example of where this may be useful is for a color lookup table
208 where we want to map the values of an image into RGB triples for
209 display. The lookup table could have a shape (nlookup, 3). Indexing
210 such an array with an image with shape (ny, nx) with dtype=np.uint8
211 (or any integer type so long as values are with the bounds of the
212 lookup table) will result in an array of shape (ny, nx, 3) where a
213 triple of RGB values is associated with each pixel location.
214
215 In general, the shape of the resulant array will be the concatenation
216 of the shape of the index array (or the shape that all the index arrays
217 were broadcast to) with the shape of any unused dimensions (those not
218 indexed) in the array being indexed.
219
220 Boolean or "mask" index arrays
221 ==============================
222
223 Boolean arrays used as indices are treated in a different manner
224 entirely than index arrays. Boolean arrays must be of the same shape
225 as the initial dimensions of the array being indexed. In the
226 most straightforward case, the boolean array has the same shape: ::
227
228 >>> b = y>20
229 >>> y[b]
230 array([21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34])
231
232 The result is a 1-D array containing all the elements in the indexed
233 array corresponding to all the true elements in the boolean array. As
234 with index arrays, what is returned is a copy of the data, not a view
235 as one gets with slices.
236
237 The result will be multidimensional if y has more dimensions than b.
238 For example: ::
239
240 >>> b[:,5] # use a 1-D boolean whose first dim agrees with the first dim of y
241 array([False, False, False, True, True], dtype=bool)
242 >>> y[b[:,5]]
243 array([[21, 22, 23, 24, 25, 26, 27],
244 [28, 29, 30, 31, 32, 33, 34]])
245
246 Here the 4th and 5th rows are selected from the indexed array and
247 combined to make a 2-D array.
248
249 In general, when the boolean array has fewer dimensions than the array
250 being indexed, this is equivalent to y[b, ...], which means
251 y is indexed by b followed by as many : as are needed to fill
252 out the rank of y.
253 Thus the shape of the result is one dimension containing the number
254 of True elements of the boolean array, followed by the remaining
255 dimensions of the array being indexed.
256
257 For example, using a 2-D boolean array of shape (2,3)
258 with four True elements to select rows from a 3-D array of shape
259 (2,3,5) results in a 2-D result of shape (4,5): ::
260
261 >>> x = np.arange(30).reshape(2,3,5)
262 >>> x
263 array([[[ 0, 1, 2, 3, 4],
264 [ 5, 6, 7, 8, 9],
265 [10, 11, 12, 13, 14]],
266 [[15, 16, 17, 18, 19],
267 [20, 21, 22, 23, 24],
268 [25, 26, 27, 28, 29]]])
269 >>> b = np.array([[True, True, False], [False, True, True]])
270 >>> x[b]
271 array([[ 0, 1, 2, 3, 4],
272 [ 5, 6, 7, 8, 9],
273 [20, 21, 22, 23, 24],
274 [25, 26, 27, 28, 29]])
275
276 For further details, consult the numpy reference documentation on array indexing.
277
278 Combining index arrays with slices
279 ==================================
280
281 Index arrays may be combined with slices. For example: ::
282
283 >>> y[np.array([0,2,4]),1:3]
284 array([[ 1, 2],
285 [15, 16],
286 [29, 30]])
287
288 In effect, the slice is converted to an index array
289 np.array([[1,2]]) (shape (1,2)) that is broadcast with the index array
290 to produce a resultant array of shape (3,2).
291
292 Likewise, slicing can be combined with broadcasted boolean indices: ::
293
294 >>> y[b[:,5],1:3]
295 array([[22, 23],
296 [29, 30]])
297
298 Structural indexing tools
299 =========================
300
301 To facilitate easy matching of array shapes with expressions and in
302 assignments, the np.newaxis object can be used within array indices
303 to add new dimensions with a size of 1. For example: ::
304
305 >>> y.shape
306 (5, 7)
307 >>> y[:,np.newaxis,:].shape
308 (5, 1, 7)
309
310 Note that there are no new elements in the array, just that the
311 dimensionality is increased. This can be handy to combine two
312 arrays in a way that otherwise would require explicitly reshaping
313 operations. For example: ::
314
315 >>> x = np.arange(5)
316 >>> x[:,np.newaxis] + x[np.newaxis,:]
317 array([[0, 1, 2, 3, 4],
318 [1, 2, 3, 4, 5],
319 [2, 3, 4, 5, 6],
320 [3, 4, 5, 6, 7],
321 [4, 5, 6, 7, 8]])
322
323 The ellipsis syntax maybe used to indicate selecting in full any
324 remaining unspecified dimensions. For example: ::
325
326 >>> z = np.arange(81).reshape(3,3,3,3)
327 >>> z[1,...,2]
328 array([[29, 32, 35],
329 [38, 41, 44],
330 [47, 50, 53]])
331
332 This is equivalent to: ::
333
334 >>> z[1,:,:,2]
335 array([[29, 32, 35],
336 [38, 41, 44],
337 [47, 50, 53]])
338
339 Assigning values to indexed arrays
340 ==================================
341
342 As mentioned, one can select a subset of an array to assign to using
343 a single index, slices, and index and mask arrays. The value being
344 assigned to the indexed array must be shape consistent (the same shape
345 or broadcastable to the shape the index produces). For example, it is
346 permitted to assign a constant to a slice: ::
347
348 >>> x = np.arange(10)
349 >>> x[2:7] = 1
350
351 or an array of the right size: ::
352
353 >>> x[2:7] = np.arange(5)
354
355 Note that assignments may result in changes if assigning
356 higher types to lower types (like floats to ints) or even
357 exceptions (assigning complex to floats or ints): ::
358
359 >>> x[1] = 1.2
360 >>> x[1]
361 1
362 >>> x[1] = 1.2j
363 <type 'exceptions.TypeError'>: can't convert complex to long; use
364 long(abs(z))
365
366
367 Unlike some of the references (such as array and mask indices)
368 assignments are always made to the original data in the array
369 (indeed, nothing else would make sense!). Note though, that some
370 actions may not work as one may naively expect. This particular
371 example is often surprising to people: ::
372
373 >>> x = np.arange(0, 50, 10)
374 >>> x
375 array([ 0, 10, 20, 30, 40])
376 >>> x[np.array([1, 1, 3, 1])] += 1
377 >>> x
378 array([ 0, 11, 20, 31, 40])
379
380 Where people expect that the 1st location will be incremented by 3.
381 In fact, it will only be incremented by 1. The reason is because
382 a new array is extracted from the original (as a temporary) containing
383 the values at 1, 1, 3, 1, then the value 1 is added to the temporary,
384 and then the temporary is assigned back to the original array. Thus
385 the value of the array at x[1]+1 is assigned to x[1] three times,
386 rather than being incremented 3 times.
387
388 Dealing with variable numbers of indices within programs
389 ========================================================
390
391 The index syntax is very powerful but limiting when dealing with
392 a variable number of indices. For example, if you want to write
393 a function that can handle arguments with various numbers of
394 dimensions without having to write special case code for each
395 number of possible dimensions, how can that be done? If one
396 supplies to the index a tuple, the tuple will be interpreted
397 as a list of indices. For example (using the previous definition
398 for the array z): ::
399
400 >>> indices = (1,1,1,1)
401 >>> z[indices]
402 40
403
404 So one can use code to construct tuples of any number of indices
405 and then use these within an index.
406
407 Slices can be specified within programs by using the slice() function
408 in Python. For example: ::
409
410 >>> indices = (1,1,1,slice(0,2)) # same as [1,1,1,0:2]
411 >>> z[indices]
412 array([39, 40])
413
414 Likewise, ellipsis can be specified by code by using the Ellipsis
415 object: ::
416
417 >>> indices = (1, Ellipsis, 1) # same as [1,...,1]
418 >>> z[indices]
419 array([[28, 31, 34],
420 [37, 40, 43],
421 [46, 49, 52]])
422
423 For this reason it is possible to use the output from the np.where()
424 function directly as an index since it always returns a tuple of index
425 arrays.
426
427 Because the special treatment of tuples, they are not automatically
428 converted to an array as a list would be. As an example: ::
429
430 >>> z[[1,1,1,1]] # produces a large array
431 array([[[[27, 28, 29],
432 [30, 31, 32], ...
433 >>> z[(1,1,1,1)] # returns a single value
434 40
435
436 """
437 from __future__ import division, absolute_import, print_function