All research

Medical imaging · 2023

Automated penile curvature measurement with deep learning

Measuring penile curvature from 2D images with a localisation, segmentation and key-point pipeline, tested on 3D-printed models and compared with a clinical expert.

Qatar University and Sidra Medicine · Hamad Medical Corporation Medical Research Center grant 20841

Pipeline diagram. A photograph of a 3D-printed model is localised and cropped, the shaft is segmented, key points are detected, and the curvature angle is calculated. A second approach detects key points on the cropped image directly.
Figure 1 from Baray et al. (2023). Two measurement approaches: key points from the segmented shaft, or key points from the cropped image. Both end in an angle calculation.View full-size figureSource articleCC BY 4.0

Approach

Can deep learning measure penile curvature from an ordinary image more consistently than visual assessment and manual tools?

Congenital penile curvature often accompanies hypospadias, which affects about 1 in 300 newborn boys, and the degree of curvature influences the choice of surgical repair. It is usually judged by eye, with a goniometer or with a phone application, and these measurements vary between observers. This study built an automated pipeline that measures the angle from a 2D image, replacing an earlier hard-coded angle calculation with learned anatomical key points.

  1. Localise and crop
  2. Segment the shaft
  3. Detect key points and calculate the angle

From a photograph to a curvature angle

Three deep learning stages

YOLOv5 first finds and crops the penile region. A UNet-based network then segments the shaft, and an HRNet model places four key points that mark the mid-axes of the distal and proximal shaft. The curvature is the angle between those two axes.

An earlier pipeline calculated the angle directly from the mask with hard-coded geometry, which failed on irregular shapes and near 90°. Key points avoid this and work for both hinge-type and arc-type curvature. Localisation reached 99.4% mAP at 0.5 overlap, and the best segmentation model, an ensemble UNet with a DenseNet121 encoder, reached a Dice score of 94.50%.

Localisation mAP at 0.5 overlap
99.4%
Shaft segmentation Dice
94.50%
Four columns showing a 3D-printed model and its segmentation mask: the input, three marked zones for the distal shaft, curvature and proximal shaft, the mid-axis of the distal and proximal shaft, and the four resulting key points.
Figure 4 from Baray et al. (2023). Four key points are defined from the distal and proximal zones of the shaft, on both the cropped image and its mask.View full-size figureSource articleCC BY 4.0

Accuracy on 3D-printed models

913 images · nine models · 18°–86°

Nine 3D-printed models with known angles provided a reference that is difficult to obtain from patients. Key-point detection used nine-fold cross-validation, holding out one model at a time so that each was tested by a network not trained on it. Measuring from segmentation masks was more accurate and more consistent than measuring from the original images, and more than halved the error of the earlier pipeline on the same dataset.

Mean absolute error from segmentation masks
3.81°
Images of nine 3D-printed models
913
Mean absolute error of the estimated angle across the nine models, from Baray et al. (2023).
Mean absolute error
Earlier pipeline (Abbas et al.)8.53°
Key points from images4.52°
Key points from masks3.81°
Two box plots of predicted against true curvature angles for nine models from 18° to 86°, one from images and one from masks. The mask-based estimates spread less around each true angle.
Figure 8 from Baray et al. (2023). Estimated angles for each model from the original images (left) and from the segmentation masks (right).View full-size figureSource articleCC BY 4.0

First comparison with a clinical expert

Four patient images

The model trained on 3D-printed models was then applied to shaft masks drawn by hand from four patient images. Its angles were compared with a clinical expert’s measurements made with a mobile angle application, and differed by 1.7° to 6.1°.

Curvature angles for the four patient images, from Table 5 of Baray et al. (2023). The difference is calculated from the two reported values.
Model estimateClinical expertDifference
Patient 117.75°23.82°6.07°
Patient 228.30°29.99°1.69°
Patient 364.43°69.79°5.36°
Patient 442.73°44.93°2.20°

My contribution

I assisted with the experimental work on automated curvature measurement and contributed to preparing and revising the manuscript.

Study scope

The main evaluation used 3D-printed models under controlled conditions. The patient comparison covered four images with hand-drawn masks, and camera angle, image quality and surgical field conditions can affect the result. Clinical use would need a larger patient dataset and a segmentation step validated on real anatomy.