STM32N6: DCMIPP

[Clyasenth]

Hello,

I want to use the TW9992 NTSC/PAL Video Decoder for STM32N6 to transfer an NTSC camera (720x480 resolution) to the MCU via MIPI-CSI and display it on the screen. I have attached the code below.

static void MX_DCMIPP_Init(void)
{
 
 /* USER CODE BEGIN DCMIPP_Init 0 */
 
 /* USER CODE END DCMIPP_Init 0 */
 
 DCMIPP_CSI_PIPE_ConfTypeDef pCSI_PipeConfig = {0};
 DCMIPP_CSI_ConfTypeDef pCSI_Config = {0};
 DCMIPP_PipeConfTypeDef pPipeConfig = {0};
 
 /* USER CODE BEGIN DCMIPP_Init 1 */
 
 /* USER CODE END DCMIPP_Init 1 */
 hdcmipp.Instance = DCMIPP;
 if (HAL_DCMIPP_Init(&hdcmipp) != HAL_OK)
 {
 Error_Handler();
 }
 
 /** Pipe 0 Config
 */
 pCSI_PipeConfig.DataTypeMode = DCMIPP_DTMODE_DTIDA;
 pCSI_PipeConfig.DataTypeIDA = DCMIPP_DT_RGB565;
 pCSI_PipeConfig.DataTypeIDB = DCMIPP_DT_RGB565;
 if (HAL_DCMIPP_CSI_PIPE_SetConfig(&hdcmipp, DCMIPP_PIPE0, &pCSI_PipeConfig) != HAL_OK)
 {
 Error_Handler();
 }
 pCSI_Config.PHYBitrate = DCMIPP_CSI_PHY_BT_1600;
 pCSI_Config.DataLaneMapping = DCMIPP_CSI_PHYSICAL_DATA_LANES;
 pCSI_Config.NumberOfLanes = DCMIPP_CSI_ONE_DATA_LANE;
 if (HAL_DCMIPP_CSI_SetConfig(&hdcmipp, &pCSI_Config) != HAL_OK)
 {
 Error_Handler();
 }
 pPipeConfig.FrameRate = DCMIPP_FRAME_RATE_ALL;
 pPipeConfig.PixelPipePitch = 1440;
 pPipeConfig.PixelPackerFormat = DCMIPP_PIXEL_PACKER_FORMAT_RGB565_1;
 if (HAL_DCMIPP_PIPE_SetConfig(&hdcmipp, DCMIPP_PIPE0, &pPipeConfig) != HAL_OK)
 {
 Error_Handler();
 }
 HAL_DCMIPP_CSI_SetVCConfig(&hdcmipp, 0U, DCMIPP_CSI_DT_BPP16);
 /* USER CODE BEGIN DCMIPP_Init 2 */
 
 /* USER CODE END DCMIPP_Init 2 */
 
}

if (HAL_DCMIPP_CSI_PIPE_Start(&hdcmipp, DCMIPP_PIPE0, DCMIPP_VIRTUAL_CHANNEL0 ,
 AXISRAM4_ADDRESS, DCMIPP_MODE_CONTINUOUS) != HAL_OK)
 {
 Error_Handler();
 }

while(1){
 /* Start DMA2D transfer*/
 if(HAL_DMA2D_Start_IT(&hdma2d,
 (uint32_t)AXISRAM4_ADDRESS, /* camera data */
 (uint32_t)BUFFER_ADDRESS, /* LCD frame buffer address */
 FRAME_WIDTH,
 FRAME_HEIGHT) != HAL_OK)
 {
 Error_Handler();
 }
 }

Is the parameter pCSI_Config.PHYBitrate = DCMIPP_CSI_PHY_BT_1600; correct for the camera used in our application?

pPipeConfig.PixelPipePitch = 1440;

HAL_DCMIPP_CSI_SetVCConfig(&hdcmipp, 0U, DCMIPP_CSI_DT_BPP16);

When using 16-bit, does it become 720 or 1440?

Also, Pipe 1 has an ISP, but we don't need to use it (the camera has its own ISP). Therefore, we chose Pipe 0 so what would you recommend about it?

Best regards,

Egemen Aksoy

[Clyasenth]

Hello,

Is there any progress?

[Clyasenth]

Hello,

Sadly, still there is no any response.

Could you please help for this topic. It is a bit urgent.

Best regards,

Egemen Aksoy

[Ch_JE]

Hello ,

The PIPE0 can be used directly to dump data from the sensor without any post-processing. In this case, the PixelPipePitch value has no impact (it is mandatory only for Pipe 1 and Pipe 2).

Please ensure that the data format is properly aligned for both CSI and DCMIPP; otherwise, no data will be dumped. You can use Pipe 0 with pCSI_PipeConfig.DataTypeMode = DCMIPP_DTMODE_ALL to directly dump all the data sent from the sensor.

Finally, ensure that the synchronization signals (HSYNC, VSYNC) from the TW9992 are properly connected and configured in DCMIPP.

Could you please share the behavior you are observing with the current configuration? Also, verify that the sensor is correctly configured with the selected virtual channel.

Regards,

 

 

[Clyasenth]

Hello,

Init code:

static void MX_DCMIPP_Init(void)
{
 
 /* USER CODE BEGIN DCMIPP_Init 0 */
 
 /* USER CODE END DCMIPP_Init 0 */
 
 DCMIPP_CSI_PIPE_ConfTypeDef pCSI_PipeConfig = {0};
 DCMIPP_CSI_ConfTypeDef pCSI_Config = {0};
 DCMIPP_PipeConfTypeDef pPipeConfig = {0};
 
 /* USER CODE BEGIN DCMIPP_Init 1 */
 
 /* USER CODE END DCMIPP_Init 1 */
 hdcmipp.Instance = DCMIPP;
 if (HAL_DCMIPP_Init(&hdcmipp) != HAL_OK)
 {
 Error_Handler();
 }
 
 /** Pipe 0 Config
 */
 pCSI_PipeConfig.DataTypeMode = DCMIPP_DTMODE_ALL;
 pCSI_PipeConfig.DataTypeIDA = DCMIPP_DT_RGB565;
 pCSI_PipeConfig.DataTypeIDB = DCMIPP_DT_RGB565;
 if (HAL_DCMIPP_CSI_PIPE_SetConfig(&hdcmipp, DCMIPP_PIPE0, &pCSI_PipeConfig) != HAL_OK)
 {
 Error_Handler();
 }
 pCSI_Config.PHYBitrate = DCMIPP_CSI_PHY_BT_1000;
 pCSI_Config.DataLaneMapping = DCMIPP_CSI_PHYSICAL_DATA_LANES;
 pCSI_Config.NumberOfLanes = DCMIPP_CSI_ONE_DATA_LANE;
 if (HAL_DCMIPP_CSI_SetConfig(&hdcmipp, &pCSI_Config) != HAL_OK)
 {
 Error_Handler();
 }
 pPipeConfig.FrameRate = DCMIPP_FRAME_RATE_ALL;
 pPipeConfig.PixelPipePitch = 1440;
 pPipeConfig.PixelPackerFormat = DCMIPP_PIXEL_PACKER_FORMAT_RGB565_1;
 if (HAL_DCMIPP_PIPE_SetConfig(&hdcmipp, DCMIPP_PIPE0, &pPipeConfig) != HAL_OK)
 {
 Error_Handler();
 }
 
 /* USER CODE BEGIN DCMIPP_Init 2 */
 HAL_DCMIPP_CSI_SetVCConfig(&hdcmipp, 0U, DCMIPP_CSI_DT_BPP8);
 /* USER CODE END DCMIPP_Init 2 */
 
}

In main.c 

if (HAL_DCMIPP_CSI_PIPE_Start(&hdcmipp, DCMIPP_PIPE0, DCMIPP_VIRTUAL_CHANNEL0 ,
 /*CAMERA_BUF_ADRESS*/BUFFER_ADDRESS, DCMIPP_MODE_CONTINUOUS) != HAL_OK)
{
Error_Handler();
}

With this configuration I see double images in the screen. What could be wrong with it?

pCSI_Config.PHYBitrate = DCMIPP_CSI_PHY_BT_1000;

How should we configure this parameter (Bit rate)?

Best regards,

[Ch_JE]

Hello,

Please attach a screenshot of the screen or the display configuration. The issue is more likely related to the display rather than the DCMIPP or the sensor.

(No overrun on dcmipp or any other error ? frames received correctly ?)

Best regards.

[Clyasenth]

Hello,

Display output looks like this.

For the display side LCD 24bit RGB interface

 pLayerCfg.WindowX0 = 0;
 pLayerCfg.WindowX1 = 720;
 pLayerCfg.WindowY0 = 0;
 pLayerCfg.WindowY1 = 480;
 pLayerCfg.PixelFormat = LTDC_PIXEL_FORMAT_RGB565;//todo fix
 pLayerCfg.Alpha = 255;
 pLayerCfg.Alpha0 = 0;
 pLayerCfg.BlendingFactor1 = LTDC_BLENDING_FACTOR1_CA;
 pLayerCfg.BlendingFactor2 = LTDC_BLENDING_FACTOR2_CA;
 pLayerCfg.FBStartAdress = BUFFER_ADDRESS;
 pLayerCfg.ImageWidth = 720;
 pLayerCfg.ImageHeight = 480;
 pLayerCfg.Backcolor.Blue = 0;
 pLayerCfg.Backcolor.Green = 0;
 pLayerCfg.Backcolor.Red = 0;
 if (HAL_LTDC_ConfigLayer(&hltdc, &pLayerCfg, 0) != HAL_OK)
 {
 Error_Handler();
 }

But we initialize RGB565 to display camera output directly to the display.

LCD resolution is 800x480.

 

Also I added an error counter in to the code and run the code many times but there is no error received.

Best regards,

[Ch_JE]

Hello,

It looks correct ,

while(1){
 /* Start DMA2D transfer*/
 if(HAL_DMA2D_Start_IT(&hdma2d,
 (uint32_t)AXISRAM4_ADDRESS, /* camera data */
 (uint32_t)BUFFER_ADDRESS, /* LCD frame buffer address */
 FRAME_WIDTH,
 FRAME_HEIGHT) != HAL_OK)
 {
 Error_Handler();
 }
 }

What is the purpose of this function? It seems you might not be managing the memory correctly, which could cause the display to show incorrect data.

The DCMIPP is used to dump data directly in the memory , allowing you to use the same frame address for display."

[Ch_JE]

You are not using a secod pipe ? 

[Clyasenth]

No I don't use second pipe.

[Clyasenth]

Hello,

Could be the video is interlaced video?

How can I deinterlace the video?

Is the STM32N6's ISP supports that such application?

Best regards,

[Ch_JE]

Yes, that is possible.

To deinterlace video, you have these options:

Using Different Virtual Channels (VC) for Odd and Even Fields on a Single Pixel Pipe with the Same Data Type:
Since a single pixel pipe can process only one VC at a time, you need to reconfigure it every frame to handle odd and even fields on different VCs by:

• Updating the VC number in the configuration register (for example, DCMIPP_P1FSCR).
• Changing the output buffer address to store odd and even frames separately in memory.
• Triggering capture (CPTREQ = 1) after each reconfiguration to start capturing the frame.
  Note: These updates must be synchronized with frame timing signals like VSync and HSync to prevent data loss.

Using Two Pixel Pipes for Interlaced Video:

• Assign Pipe1 to handle the odd field (VC0) and Pipe2 to handle the even field (VC1).
• Configure each pipe with its own fixed VC number and output buffer address.
• Set PIPEDIFF = 1 to make Pipe1 and Pipe2 operate independently.
• Both pipes run simultaneously without needing frame-by-frame reconfiguration.
• Use continuous capture mode (CPTMODE = 0) and enable capture (CPTREQ = 1) on both pipes.
  This approach simplifies management and eliminates the overhead of reconfiguration.

Additionally, you can use Pipe0 together with one pixel pipe if image processing is not required in your application.

For more details see part Interlaced video inthe chapter CSI2 camera sendor module

Please add more details/datasheet about the sensor configuration 

 

 

 

[Clyasenth]

Hello,

If I use second method (Using Two Pixel Pipes for Interlaced Video) and for example assigning even ones into a x address and even ones into the y address how can I use them to obtain single (progressive) image in our display?

Best regards, 

[Ch_JE]

To get a full frame, you need to merge these two fields line by line:

• Allocate a new output buffer for the progressive frame.
• For each line in the output buffer:
  • Copy the line from the odd field buffer if the line number is odd.
  • Copy the line from the even field buffer if the line number is even.

[Clyasenth]

Hello,

Could I use HPDMA and/or GPDMA for transferring the frame buffer to deinterlace the image?

Can this application be implemented without overloading the MCU?

Best regards,

[Ch_JE]

Yes, sure, you can use the DMA to transfer the data from the Odd/Even destinations to the new output buffer dedicated for the progressive frame instead of overloading the MCU.

For synchronization, you can use the dedicated DCMIPP - GPDMA or HPDMA triggers available for the events generated by the DCMIPP (lines, frames, VSYNC, etc.). 

See HPDMA/GPDMA triggers sections in the referance Manual