opencv_geometry

OpenCV，几何问题 §

1 形状的表示方式 §

1.1 矩形 §

• 矩形可以用多变形/四个顶角（contour）或中心+长宽+旋转角度（box2d）两种形式进行表示

1.1.1 多变形形式 §

• 获取某个位置的顶角坐标（如最高点坐标），也可以用不带旋转的选定框间接获取。注意输入的数据poly其类型应为二维nparray

  sort_y = poly[poly[:,1].argsort()]
  sort_x = poly[poly[:,0].argsort()]
  y1 = sort_y[0,1]
  y2 = sort_y[-1,1]
  x1 = sort_x[0,0]
  x2 = sort_x[-1,0]

1.1.2 中心点与长宽、旋转的形式 §

• 包含了矩形的中心、长宽、相对于水平轴的角度((x,y),(w,h),angle)
• 转换为多边形顶角：cv2.boxPoints()

rect = np.int0(cv2.boxPoints(rect))
cv2.drawContours(img, [rect], 0, (0, 0, 255), 4)  # green

• box2d数据类型转换为用四点坐标表示矩形的contour类型，先用boxPoints()转换为浮点型的坐标，再强制类型转换为int：

  box_contour = np.int0(cv2.boxPoints(min_rect))

2 绘制图形 §

2.1 用polylines绘制多边形（如四边形） §

cv2.polylines(img, [np.array([[0,0],[3,0],[3,2],[0,2]])], isClosed=True, color=(0, 0, 255),thickness=2)

3 轮廓Contour §

3.1 从图像中提取轮廓 ^extract-contour §

• 代码示例

def find_all_contours(img, size_min=50):
    all_contours, hierarchy = cv2.findContours(img,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
    contours = []
    for i in range(len(all_contours)):
        area = cv2.contourArea(all_contours[i])
        if area > size_min:
            contours.append(i)
 
    cont = [all_contours[i] for i in contours]
    return cont

3.2 轮廓的重心和面积 §

m = cv2.moments(contour)
cx = int(m["m10"] / m["m00"])
cy = int(m["m01"] / m["m00"])
area = cv2.contourArea(contour)

3.3 绘制轮廓（注意和绘制四边形不一样！但也可以用于绘制多边形） §

• 使用cv2.drawContours()
  • 传入的第二个参量contours是一个list，包含若干个外形。每一个外形都是一个维度为(n,1,2)的矩阵（numpy.array）
  • 第二个参量为绘制第一个外形，-1为全部，0为第一个

cv2.drawContours(img, contours, -1, (0, 0, 255),2)

使用drawContours()绘制多边形时，多边形的顶点传入时用方括号包裹，例

cv2.drawContours(img, [np.array([[0,0],[3,0],[3,2],[0,2]])], 0, (0, 0, 255),2)

3.4 填充轮廓 §

• contours为上面从图像中提取轮廓find_all_contours的返回值，即一个列表，列表中的每个元素为一个轮廓

cv2.fillPoly(img, pts=contours, color=255)

3.5 合并多个轮廓 §

• 参见多个contour合并为一个hull

3.6 Mask的最大外接多边形与最大内接四边形 §

3.6.1 包裹给定目标的最小外接长方形（带旋转角度）minAreaRect() §

• cv2.minAeraRect() 一般需要先生成一个轮廓列表，然后逐一检查轮廓，为其附以相应的矩形 输入为一个单一轮廓（一维列表，不能是轮廓列表/二位列表） 输出为矩形的中心(x,y)，大小(w,h)，旋转角度，注意不同OpenCV版本中的旋转方式可能有所不同！ OpenCV官方文档中描述矩形使用的是width and height

  width and height of the rectangle

for cont in contours:
    min_rect = cv2.minAreaRect(cont)  # min_area_rectangle
    min_rect = np.int0(cv2.boxPoints(min_rect)) # turn the ((x,y),(w,h),angle) to the position of four corners
    cv2.drawContours(img, [min_rect], 0, (0, 0, 255), 4)

3.6.2 填充给定目标的最大内接长方形（带旋转角度） §

import numpy as np
import cv2
import copy
 
## https://stackoverflow.com/a/30418912/5008845
## Find largest rectangle containing only zeros in an N×N binary matrix
def find_min_rect(src, threshold=100):
    # return: top left corner and bottom right corner
    # [x0, y0, x1, y1]
 
    rows = src.shape[0]
    cols = src.shape[1]
    W = np.zeros([rows, cols])
    H = np.zeros([rows, cols])
 
    maxRect = [0,0,0,0]
    maxArea = .0
 
    for r in range(rows):
        for c in range(cols):
            if src[r, c] > threshold:
                i = 0
                if r > 0:
                    i = H[r-1, c]
 
                H[r, c] = 1.0 + i
 
                j = 0
                if c > 0:
                    j = W[r, c-1]
 
                W[r, c] = 1.0 +j
 
            minw = W[r,c];
            for i in range(int(H[r,c])):
                minw = min(minw, W[r-i, c])
                area = (i+1) * minw;
                if area > maxArea:
                    maxArea = area;
                    maxRect = [c - minw + 1, r - i, c+1, r+1]
 
    return [int(i) for i in maxRect]
 
def area(box):
    return (box[2]-box[0])*(box[3]-box[1])
 
def find_rect_w_rotation(src, angle):
    maxdim = src.shape[0]//2
    ## Rotate the image
    m = cv2.getRotationMatrix2D((maxdim,maxdim), angle, 1)
    # Mat1b rotated
    rotated = cv2.warpAffine(src, m, src.shape)
 
    ## Keep the crop with the polygon
    pts = cv2.findNonZero(rotated)
    box = cv2.boundingRect(pts)
    x = box[0]
    y = box[1]
    w = box[2]
    h = box[3]
    crop = rotated[y:y+h,x:x+w]
 
    ## Solve the problem: "Find largest rectangle containing only zeros in an binary matrix"
    ## https://stackoverflow.com/questions/2478447/find-largest-rectangle-containing-only-zeros-in-an-n%C3%97n-binary-matrix
    p = find_min_rect(crop);
 
    return [p[0]+x, p[1]+y, p[2]+x, p[3]+y]
 
 
def find_best_angle(src, size=-1):
    # The input should be a square image
    if size == -1:
        work = src
        maxdim = work.shape[0]
        ...
    else:
        maxdim = size*2
        # resize image
        work =  cv2.resize(src, (maxdim, maxdim), interpolation = cv2.INTER_AREA)
 
    # ## Store best data
    bestRect = [0,0,0,0]
    bestAngle = 0 # int
 
    ## For each angle
    # for angle in range(90):
    for angle in range(-30, 30):
        p = find_rect_w_rotation(work, angle)
        ## If best, save result
        if (area(p) > area(bestRect)):
            # Correct the crop displacement
            # bestRect = r + box.tl();
            bestRect = [p[0], p[1], p[2], p[3]]
            bestAngle = angle
 
    return bestAngle
 
## https://stackoverflow.com/questions/32674256
## How to adapt or resize a rectangle inside an object without including (or with a few numbers) of background pixels?
def largest_rect_in_non_convex_poly(src, thumbnail_size = -1):
    ## Create a matrix big enough to not lose points during rotation
    ptz = cv2.findNonZero(src)
    bbox = cv2.boundingRect(ptz) #bbox is [x, y, w, h]
    x = bbox[0]
    y = bbox[1] 
    width = bbox[2]
    height = bbox[3]
 
    if width%2==1:
        width -= 1
    ## height += 1 may leads to a y+height > 720 case 
    if height%2==1:
        height -= 1
 
    maxdim = max(width, height)
    work = np.zeros([2*maxdim, 2*maxdim])
    work[maxdim-height//2:maxdim+height//2,maxdim-width//2:maxdim+width//2] \
        = copy.deepcopy(src[y:y+height,x:x+width])
 
    # use a thumbnail to accelerate finding the best angle first
    # or set the size = -1 to use the original image to find the best angle
    bestAngle = find_best_angle(work, thumbnail_size)
    # use the original image and the found best angle to find the largest rectangle
    bestRect = find_rect_w_rotation(work, bestAngle)
 
    ## Apply the inverse rotation
    m_inv = cv2.getRotationMatrix2D((maxdim, maxdim), -bestAngle, 1)
    x0 = bestRect[0]
    y0 = bestRect[1]
    x1 = bestRect[2]
    y1 = bestRect[3]
    ## OpenCV on Python often wants points in the form
    ## np.array([ [[x1, y1]], ..., [[xn, yn]] ])
    rectPoints = np.array([ [[x0, y0]], [[x1, y0]], [[x1, y1]], [[x0, y1]] ])
    rotatedRectPoints = cv2.transform(rectPoints, m_inv)
 
    ## Apply the reverse translations
    for i in range(len(rotatedRectPoints)):
        ## rotatedRectPoints += bbox.tl() - Point(maxdim - bbox.width / 2, maxdim - bbox.height / 2)
        rotatedRectPoints[i][0][0] += x - maxdim + width / 2
        rotatedRectPoints[i][0][1] += y - maxdim + height / 2
 
    ## Get the rotated rect
    ## (center(x, y), (width, height), angle of rotation) = cv2.minAreaRect(points)
    rrect = cv2.minAreaRect(rotatedRectPoints)
 
    return rrect
 
 
if __name__ == '__main__':
    img = cv2.imread("./mask.png", cv2.IMREAD_GRAYSCALE)
 
    # Compute largest rect inside polygon
    rect = largest_rect_in_non_convex_poly(img, thumbnail_size=100)
    print(rect)
    box = cv2.boxPoints(rect)
    box = np.int0(box)
    res = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
    cv2.drawContours(res,[box],0,(0,0,255),2)
    cv2.imshow("Result", res)
    cv2.waitKey()
 
    # p = find_min_rect(img)
    # print(p)
    # r = [[p[0],p[1]],[p[0],p[3]],[p[2],p[3]],[p[2],p[1]]]
    # res = cv2.cvtColor(img, cv2.COLOR_GRAY2BGR)
    # for i in range(len(r)):
    #     cv2.line(res, r[i], r[(i+1)%4], (0, 0, 255), 2)
 
    # cv2.imshow("Result", res)
    # cv2.waitKey()

3.6.3 检测点是否在多边形内 §

• 三个参数分别为多边形、待检测的点，及返回数据类型。如返回数据类型为False，则-1表示在多边形外，0表示在多边形上，1表示在多边形内。否则返回数据为点距离多边形的距离，负数表示在多边形外，正数表示在多边形内。

polygon = [[506, 195], [995, 179], [997, 248], [508, 265]]
pt = (100,40)
dist = cv2.pointPolygonTest(np.array(polygon), pt, False)

4 凸包ConvexHull §

4.1.1 Hull的数据类型 §

• Hull的数据类型是一个顶点列表，相当于一个多边形（polyon）

4.1.2 绘制ConvexHull的外形 §

• 注意用[ ]将其转换为列表，否则画出来的只是几个角点 cv2.drawContours(contour2_img, [hull],-1,255,1)

4.1.3 Hull的填充 §

• “cv2.fillConvexPoly(img, hull, 255) → None`

4.1.4 多个contour合并为一个hull ^merge-contour §

• 输入为一个包含所有contours的列表（参考从图像中提取轮廓)

def get_single_hull(contours):
    cont = np.vstack([i for i in contours])
    hull = cv2.convexHull(cont)
    return hull

5 拓扑 §

5.1 提取骨架 §

• 使用scikit-image的skeletonize?
  • https://scikit-image.org/docs/stable/auto_examples/edges/plot_skeleton.html from skimage.morphology import skeletonize
  • 代码示例（转换时注意数据类型的变换，需要把OpenCV Image类型转换为skimage类型，参考）：

  from skimage.morphology import skeletonize
   
  ## mask is a numpy array taht contains only 0s and 255s
  flesh = np.where(mask>100, 1, 0)
  skeleton = skeletonize(flesh)
  output = (np.where(skeleton==True, 255, 0)).astype(np.uint8)

• 使用scikit-image的mdeial axis skeletonization?
  • from skimage.morphology import medial_axis