summaryrefslogtreecommitdiff
path: root/drivers/staging/fsl-mc/README.txt
blob: 8214102f104bd46239bf696bec8a377deed89124 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
Copyright (C) 2015 Freescale Semiconductor Inc.

DPAA2 (Data Path Acceleration Architecture Gen2)
------------------------------------------------

This document provides an overview of the Freescale DPAA2 architecture
and how it is integrated into the Linux kernel.

Contents summary
   -DPAA2 overview
   -Overview of DPAA2 objects
   -DPAA2 Linux driver architecture overview
        -bus driver
        -dprc driver
        -allocator
        -dpio driver
        -Ethernet
        -mac

DPAA2 Overview
--------------

DPAA2 is a hardware architecture designed for high-speeed network
packet processing.  DPAA2 consists of sophisticated mechanisms for
processing Ethernet packets, queue management, buffer management,
autonomous L2 switching, virtual Ethernet bridging, and accelerator
(e.g. crypto) sharing.

A DPAA2 hardware component called the Management Complex (or MC) manages the
DPAA2 hardware resources.  The MC provides an object-based abstraction for
software drivers to use the DPAA2 hardware.

The MC uses DPAA2 hardware resources such as queues, buffer pools, and
network ports to create functional objects/devices such as network
interfaces, an L2 switch, or accelerator instances.

The MC provides memory-mapped I/O command interfaces (MC portals)
which DPAA2 software drivers use to operate on DPAA2 objects:

         +--------------------------------------+
         |                  OS                  |
         |                        DPAA2 drivers |
         |                             |        |
         +-----------------------------|--------+
                                       |
                                       | (create,discover,connect
                                       |  config,use,destroy)
                                       |
                         DPAA2         |
         +------------------------| mc portal |-+
         |                             |        |
         |   +- - - - - - - - - - - - -V- - -+  |
         |   |                               |  |
         |   |   Management Complex (MC)     |  |
         |   |                               |  |
         |   +- - - - - - - - - - - - - - - -+  |
         |                                      |
         | Hardware                  Hardware   |
         | Resources                 Objects    |
         | ---------                 -------    |
         | -queues                   -DPRC      |
         | -buffer pools             -DPMCP     |
         | -Eth MACs/ports           -DPIO      |
         | -network interface        -DPNI      |
         |  profiles                 -DPMAC     |
         | -queue portals            -DPBP      |
         | -MC portals                ...       |
         |  ...                                 |
         |                                      |
         +--------------------------------------+

The MC mediates operations such as create, discover,
connect, configuration, and destroy.  Fast-path operations
on data, such as packet transmit/receive, are not mediated by
the MC and are done directly using memory mapped regions in
DPIO objects.

Overview of DPAA2 Objects
-------------------------
The section provides a brief overview of some key objects
in the DPAA2 hardware.  A simple scenario is described illustrating
the objects involved in creating a network interfaces.

-DPRC (Datapath Resource Container)

    A DPRC is an container object that holds all the other
    types of DPAA2 objects.  In the example diagram below there
    are 8 objects of 5 types (DPMCP, DPIO, DPBP, DPNI, and DPMAC)
    in the container.

    +---------------------------------------------------------+
    | DPRC                                                    |
    |                                                         |
    |  +-------+  +-------+  +-------+  +-------+  +-------+  |
    |  | DPMCP |  | DPIO  |  | DPBP  |  | DPNI  |  | DPMAC |  |
    |  +-------+  +-------+  +-------+  +---+---+  +---+---+  |
    |  | DPMCP |  | DPIO  |                                   |
    |  +-------+  +-------+                                   |
    |  | DPMCP |                                              |
    |  +-------+                                              |
    |                                                         |
    +---------------------------------------------------------+

    From the point of view of an OS, a DPRC is bus-like.  Like
    a plug-and-play bus, such as PCI, DPRC commands can be used to
    enumerate the contents of the DPRC, discover the hardware
    objects present (including mappable regions and interrupts).

     dprc.1 (bus)
       |
       +--+--------+-------+-------+-------+
          |        |       |       |       |
        dpmcp.1  dpio.1  dpbp.1  dpni.1  dpmac.1
        dpmcp.2  dpio.2
        dpmcp.3

    Hardware objects can be created and destroyed dynamically, providing
    the ability to hot plug/unplug objects in and out of the DPRC.

    A DPRC has a mappable mmio region (an MC portal) that can be used
    to send MC commands.  It has an interrupt for status events (like
    hotplug).

    All objects in a container share the same hardware "isolation context".
    This means that with respect to an IOMMU the isolation granularity
    is at the DPRC (container) level, not at the individual object
    level.

    DPRCs can be defined statically and populated with objects
    via a config file passed to the MC when firmware starts
    it.  There is also a Linux user space tool called "restool"
    that can be used to create/destroy containers and objects
    dynamically.

-DPAA2 Objects for an Ethernet Network Interface

    A typical Ethernet NIC is monolithic-- the NIC device contains TX/RX
    queuing mechanisms, configuration mechanisms, buffer management,
    physical ports, and interrupts.  DPAA2 uses a more granular approach
    utilizing multiple hardware objects.  Each object has specialized
    functions, and are used together by software to provide Ethernet network
    interface functionality.  This approach provides efficient use of finite
    hardware resources, flexibility, and performance advantages.

    The diagram below shows the objects needed for a simple
    network interface configuration on a system with 2 CPUs.

              +---+---+ +---+---+
                 CPU0     CPU1
              +---+---+ +---+---+
                  |         |
              +---+---+ +---+---+
                 DPIO     DPIO
              +---+---+ +---+---+
                    \     /
                     \   /
                      \ /
                   +---+---+
                      DPNI  --- DPBP,DPMCP
                   +---+---+
                       |
                       |
                   +---+---+
                     DPMAC
                   +---+---+
                       |
                    port/PHY

    Below the objects are described.  For each object a brief description
    is provided along with a summary of the kinds of operations the object
    supports and a summary of key resources of the object (mmio regions
    and irqs).

       -DPMAC (Datapath Ethernet MAC): represents an Ethernet MAC, a
        hardware device that connects to an Ethernet PHY and allows
        physical transmission and reception of Ethernet frames.
           -mmio regions: none
           -irqs: dpni link change
           -commands: set link up/down, link config, get stats,
             irq config, enable, reset

       -DPNI (Datapath Network Interface): contains TX/RX queues,
        network interface configuration, and rx buffer pool configuration
        mechanisms.
           -mmio regions: none
           -irqs: link state
           -commands: port config, offload config, queue config,
            parse/classify config, irq config, enable, reset

       -DPIO (Datapath I/O): provides interfaces to enqueue and dequeue
        packets and do hardware buffer pool management operations.  For
        optimum performance there is typically DPIO per CPU.  This allows
        each CPU to perform simultaneous enqueue/dequeue operations.
           -mmio regions: queue operations, buffer mgmt
           -irqs: data availability, congestion notification, buffer
                  pool depletion
           -commands: irq config, enable, reset

       -DPBP (Datapath Buffer Pool): represents a hardware buffer
        pool.
           -mmio regions: none
           -irqs: none
           -commands: enable, reset

       -DPMCP (Datapath MC Portal): provides an MC command portal.
        Used by drivers to send commands to the MC to manage
        objects.
           -mmio regions: MC command portal
           -irqs: command completion
           -commands: irq config, enable, reset

    Object Connections
    ------------------
    Some objects have explicit relationships that must
    be configured:

       -DPNI <--> DPMAC
       -DPNI <--> DPNI
       -DPNI <--> L2-switch-port
          A DPNI must be connected to something such as a DPMAC,
          another DPNI, or L2 switch port.  The DPNI connection
          is made via a DPRC command.

              +-------+  +-------+
              | DPNI  |  | DPMAC |
              +---+---+  +---+---+
                  |          |
                  +==========+

       -DPNI <--> DPBP
          A network interface requires a 'buffer pool' (DPBP
          object) which provides a list of pointers to memory
          where received Ethernet data is to be copied.  The
          Ethernet driver configures the DPBPs associated with
          the network interface.

    Interrupts
    ----------
    All interrupts generated by DPAA2 objects are message
    interrupts.  At the hardware level message interrupts
    generated by devices will normally have 3 components--
    1) a non-spoofable 'device-id' expressed on the hardware
    bus, 2) an address, 3) a data value.

    In the case of DPAA2 devices/objects, all objects in the
    same container/DPRC share the same 'device-id'.
    For ARM-based SoC this is the same as the stream ID.


DPAA2 Linux Driver Overview
---------------------------

This section provides an overview of the Linux kernel drivers for
DPAA2-- 1) the bus driver and associated "DPAA2 infrastructure"
drivers and 2) functional object drivers (such as Ethernet).

As described previously, a DPRC is a container that holds the other
types of DPAA2 objects.  It is functionally similar to a plug-and-play
bus controller.

Each object in the DPRC is a Linux "device" and is bound to a driver.
The diagram below shows the Linux drivers involved in a networking
scenario and the objects bound to each driver.  A brief description
of each driver follows.

                                             +------------+
                                             | OS Network |
                                             |   Stack    |
                 +------------+              +------------+
                 | Allocator  |. . . . . . . |  Ethernet  |
                 |(dpmcp,dpbp)|              |   (dpni)   |
                 +-.----------+              +---+---+----+
                  .          .                   ^   |
                 .            .     <data avail, |   |<enqueue,
                .              .     tx confirm> |   | dequeue>
    +-------------+             .                |   |
    | DPRC driver |              .           +---+---V----+     +---------+
    |   (dprc)    |               . . . . . .| DPIO driver|     |   MAC   |
    +----------+--+                          |  (dpio)    |     | (dpmac) |
               |                             +------+-----+     +-----+---+
               |<dev add/remove>                    |                 |
               |                                    |                 |
          +----+--------------+                     |              +--+---+
          |   mc-bus driver   |                     |              | PHY  |
          |                   |                     |              |driver|
          | /fsl-mc@80c000000 |                     |              +--+---+
          +-------------------+                     |                 |
                                                    |                 |
 ================================ HARDWARE =========|=================|======
                                                  DPIO                |
                                                    |                 |
                                                  DPNI---DPBP         |
                                                    |                 |
                                                  DPMAC               |
                                                    |                 |
                                                   PHY ---------------+
 ===================================================|========================

A brief description of each driver is provided below.

    mc-bus driver
    -------------
    The mc-bus driver is a platform driver and is probed from an
    "/fsl-mc@xxxx" node in the device tree passed in by boot firmware.
    It is responsible for bootstrapping the DPAA2 kernel infrastructure.
    Key functions include:
       -registering a new bus type named "fsl-mc" with the kernel,
        and implementing bus call-backs (e.g. match/uevent/dev_groups)
       -implemeting APIs for DPAA2 driver registration and for device
        add/remove
       -creates an MSI irq domain
       -do a device add of the 'root' DPRC device, which is needed
        to bootstrap things

    DPRC driver
    -----------
    The dprc-driver is bound DPRC objects and does runtime management
    of a bus instance.  It performs the initial bus scan of the DPRC
    and handles interrupts for container events such as hot plug.

    Allocator
    ----------
    Certain objects such as DPMCP and DPBP are generic and fungible,
    and are intended to be used by other drivers.  For example,
    the DPAA2 Ethernet driver needs:
       -DPMCPs to send MC commands, to configure network interfaces
       -DPBPs for network buffer pools

    The allocator driver registers for these allocatable object types
    and those objects are bound to the allocator when the bus is probed.
    The allocator maintains a pool of objects that are available for
    allocation by other DPAA2 drivers.

    DPIO driver
    -----------
    The DPIO driver is bound to DPIO objects and provides services that allow
    other drivers such as the Ethernet driver to receive and transmit data.
    Key services include:
        -data availability notifications
        -hardware queuing operations (enqueue and dequeue of data)
        -hardware buffer pool management

    There is typically one DPIO object per physical CPU for optimum
    performance, allowing each CPU to simultaneously enqueue
    and dequeue data.

    The DPIO driver operates on behalf of all DPAA2 drivers
    active in the kernel--  Ethernet, crypto, compression,
    etc.

    Ethernet
    --------
    The Ethernet driver is bound to a DPNI and implements the kernel
    interfaces needed to connect the DPAA2 network interface to
    the network stack.

    Each DPNI corresponds to a Linux network interface.

    MAC driver
    ----------
    An Ethernet PHY is an off-chip, board specific component and is managed
    by the appropriate PHY driver via an mdio bus.  The MAC driver
    plays a role of being a proxy between the PHY driver and the
    MC.  It does this proxy via the MC commands to a DPMAC object.