Generative Image Processing: Edge-Based Vector Fields
The original image is the photo of the Gargano National Park.
Pushing the Boundaries of Image Transformation
This project explores a unique approach to generative image processing using Processing. By combining traditional edge detection with custom vector fields, I've developed a method that "re-draws" a photograph through organic, algorithmic strokes.
Following the concepts explored in my previous work 'Daydream', this algorithm seeds a vector field at the high-contrast edges of an image, allowing the photo's inherent structure to guide the generative flow.
You can use your photo to process.
PImage img = loadImage("your_photo.png");
Key Algorithmic Concepts: Nested Noise and Quantized Angles
The core of this effect lies in the way the vector field is calculated. Instead of a standard 2D noise(x, y), I utilized nested 3D noise (often referred to as domain warping):
noise(xPrev * _noiseDiv, yPrev * _noiseDiv, noise(yPrev * _noiseDiv * 3.0) * _uneven)
Ordinary, I use 2D noise like 'noise(x, y)'.
When I used nested 3D noise like 'noise(x, y, noise(x, y))', it creates a subtle "unevenness" and structural complexity that simple noise cannot achieve.
Another crucial element is the quantization of flow angles. By rounding the directions of the paths, I can control the texture of the result—ranging from smooth, natural flows to rigid, geometric patterns.
cos(TWO_PI * round(nX * _curlMult * _angles) / _angles)
An example image : no limitting.
An example image : limitting the direction.
The Implementation
The following script is designed for offline rendering, saving high-resolution frames directly to your directory. It serves as a framework for transforming any photo into a piece of generative art.
Warning: This code process the image hard!
Please feel free to use this example code under the terms of the GPL. To see other works based on my code is my pleasure. And my honor.
/**
* Picture This.
* The Vector field that starts from the edges of some photo.
*
* @author @deconbatch
* @version 0.2
* @license GPL Version 3 http://www.gnu.org/licenses/
* Processing 4.3.3
* 0.1 2020-11-19 created
* 0.2 2026-03-02 modified to produce a more painterly result
* 0.3 2026-03-08 modified bug fix on edge detection
*/
void setup() {
size(980, 980);
colorMode(HSB, 360, 100, 100, 100);
smooth();
noLoop();
}
void draw() {
int imgMax = 3;
int detectMax = 30000;
int caseWidth = 30;
int baseCanvas = width - caseWidth * 2;
// original photo load
PImage img = loadImage("your_photo.jpg");
float rateSize = baseCanvas * 1.0 / max(img.width, img.height);
img.resize(floor(img.width * rateSize), floor(img.height * rateSize));
img.loadPixels();
int iW = img.width;
int iH = img.height;
// detect edges
ArrayList<PVector> edges = detectEdge(img, detectMax);
float initHue = typicalHue(img, edges);
for (int imgCnt = 0; imgCnt < imgMax; imgCnt++) {
float curlMult = random(2.0,10.0);
float noiseDiv = 0.001;
float uneven = random(5.0, 10.0);
int angles = floor(5 + random(4.0) * 2.0);
float baseSiz = 1.5;
// calculate the vector field
ArrayList<ArrayList<PVector>> paths = getPaths(edges, curlMult, noiseDiv, uneven, angles);
// draw background
pushMatrix();
translate((width - iW) / 2, (height - iH) / 2);
blendMode(BLEND);
background((initHue + 180.0) % 360.0, 30.0, 40.0, 100);
noStroke();
// draw shadow
for (ArrayList<PVector> path : paths) {
float xInit = path.get(0).x;
float yInit = path.get(0).y;
float xPoint = xInit;
float yPoint = yInit;
int initIdx = floor(yPoint * iW + xPoint);
float baseHue = hue(img.pixels[initIdx]);
float baseBri = brightness(img.pixels[initIdx]);
for (int i = 0; i < path.size(); i++) {
PVector p = path.get(i);
if (
p.x >= 0 && p.x < iW &&
p.y >= 0 && p.y < iH
) {
int pathIdx = floor(constrain(p.y * iW + p.x, 0, iW * iH - 1));
float pRatio = i * 1.0 / path.size();
float eHue = baseHue + 330.0 + floor(((xInit * yInit) * 1.0) % 4.0) * 20.0;
float eBri = baseBri * sin(PI * pRatio) * (8.0 + floor(((xInit * yInit) * 20000.0) % 5.0)) / 30.0;
float eSat = 90.0;
float eSiz = baseSiz * sin(PI * pRatio);
pushMatrix();
if (brightness(img.pixels[pathIdx]) < 30.0) {
eBri = 80.0 - eBri;
eSat = 10.0;
translate(-eSiz * 2.0, -eSiz * 2.0);
} else {
translate(eSiz * 2.0, eSiz * 2.0);
}
fill(eHue % 360.0, eSat, eBri, 100);
circle(p.x, p.y, eSiz);
popMatrix();
}
}
}
// draw foreground
for (ArrayList<PVector> path : paths) {
float xInit = path.get(0).x;
float yInit = path.get(0).y;
float xPoint = xInit;
float yPoint = yInit;
int initIdx = floor(yPoint * iW + xPoint);
float baseHue = hue(img.pixels[initIdx]);
float baseBri = brightness(img.pixels[initIdx]);
for (int i = 0; i < path.size(); i++) {
PVector p = path.get(i);
if (
p.x >= 0 && p.x < iW &&
p.y >= 0 && p.y < iH
) {
int pathIdx = floor(constrain(p.y * iW + p.x, 0, iW * iH - 1));
float pRatio = i * 1.0 / path.size();
float eHue = baseHue + 330.0 + floor(((xInit * yInit) * 1.0) % 4.0) * 20.0;
float eSat = saturation(img.pixels[pathIdx]);
float eBri = 0.5 * (brightness(img.pixels[pathIdx]) + baseBri * sin(PI * pRatio) * (8.0 + floor(((xInit * yInit) * 20000.0) % 5.0)) / 10.0);
float eSiz = baseSiz * sin(PI * pRatio);
fill(eHue % 360.0, eSat, eBri, 100.0);
circle(p.x, p.y, eSiz);
}
}
}
popMatrix();
casing();
saveFrame("frames/" + String.format("%04d", imgCnt + 1) + ".png");
}
// save the original image
background((initHue + 180.0) % 360.0, 30.0, 40.0, 100);
imageMode(CENTER);
image(img, width * 0.5, height * 0.5);
casing();
saveFrame("frames/0000.png");
exit();
}
/**
* casing : draw fancy casing
*/
private void casing() {
blendMode(BLEND);
fill(0.0, 0.0, 0.0, 0.0);
strokeWeight(60.0);
stroke(0.0, 0.0, 0.0, 100.0);
rect(0.0, 0.0, width, height);
strokeWeight(50.0);
stroke(0.0, 0.0, 100.0, 100.0);
rect(0.0, 0.0, width, height);
noStroke();
noFill();
}
/**
* getPaths : calculate the Vector Field paths.
* @param _pvs : start points coordinate of the Vector Field.
* @param _curlMult : curl ratio of the path.
* @param _noiseDiv : noise parameter step ratio.
* @param _uneven : make uneven the Vector Field with 3rd parameter of the noise.
* @param _angles : slant of the path.
* @return array of the Vector Field path, path is the array of the PVector coordinates.
*/
private ArrayList<ArrayList<PVector>> getPaths(ArrayList<PVector> _pvs, float _curlMult, float _noiseDiv, float _uneven, int _angles) {
int plotMax = 2000;
float plotDiv = 0.1;
ArrayList<ArrayList<PVector>> paths = new ArrayList<ArrayList<PVector>>();
for (PVector p : _pvs) {
ArrayList<PVector> path = new ArrayList<PVector>();
float xInit = p.x;
float yInit = p.y;
float xPoint = xInit;
float yPoint = yInit;
for (int plotCnt = 0; plotCnt < plotMax; ++plotCnt) {
float xPrev = xPoint;
float yPrev = yPoint;
float nX = noise(xPrev * _noiseDiv, yPrev * _noiseDiv, noise(yPrev * _noiseDiv * 3.0) * _uneven);
float nY = noise(yPrev * _noiseDiv, xPrev * _noiseDiv, noise(xPrev * _noiseDiv * 3.0) * _uneven);
xPoint += plotDiv * cos(TWO_PI * round(nX * _curlMult * _angles) / _angles);
yPoint += plotDiv * sin(TWO_PI * round(nY * _curlMult * _angles) / _angles);
path.add(new PVector(floor(xPoint), floor(yPoint)));
}
if (dist(path.get(0).x, path.get(0).y, path.get(path.size() - 1).x, path.get(path.size() - 1).y) > 20.0) {
paths.add(path);
}
}
return paths;
}
/**
* detectEdge : detect edges of photo image with Sobel operator.
* @param _img : detect edges of this image.
* @param _edgeNum : return edge points number.
* @return array of the edges locations.
*/
private ArrayList<PVector> detectEdge(PImage _img, int _edgeNum) {
int chkDiv = 5;
ArrayList<PVector> edges = new ArrayList<PVector>();
_img.loadPixels();
for (int idxW = chkDiv; idxW < _img.width - chkDiv; idxW += chkDiv) {
for (int idxH = chkDiv; idxH < _img.height - chkDiv; idxH += chkDiv) {
int pixIndex = idxH * _img.width + idxW;
// hue difference
// Technically, hue values of 0 and 360 represent the same color. Since there's a huge numerical gap between them.
float hueC = hue(_img.pixels[pixIndex]);
float hueN = hue(_img.pixels[pixIndex - _img.width * chkDiv]);
float hueNW = hue(_img.pixels[pixIndex - _img.width * chkDiv - chkDiv]);
float hueNE = hue(_img.pixels[pixIndex - _img.width * chkDiv + chkDiv]);
float hueW = hue(_img.pixels[pixIndex - chkDiv]);
float hueE = hue(_img.pixels[pixIndex + chkDiv]);
float hueS = hue(_img.pixels[pixIndex + _img.width * chkDiv]);
float hueSW = hue(_img.pixels[pixIndex + _img.width * chkDiv - chkDiv]);
float hueSE = hue(_img.pixels[pixIndex + _img.width * chkDiv + chkDiv]);
float lapHueH =
- hueW * 2.0
- hueNW
- hueSW
+ hueE * 2.0
+ hueNE
+ hueSE;
float modHueH = lapHueH % 360.0;
if (modHueH > 180.0) {
lapHueH = modHueH - 360.0;
} else {
lapHueH = modHueH;
}
float lapHueV =
+ hueN * 2.0
+ hueNW
+ hueNE
- hueS * 2.0
- hueSW
- hueSE;
float modHueV = lapHueV % 360.0;
if (modHueV > 180.0) {
lapHueV = modHueV - 360.0;
} else {
lapHueV = modHueV;
}
float lapHue = sqrt(
pow(lapHueH, 2) +
pow(lapHueV, 2)
);
float modHue = lapHue % 360.0;
if (modHue > 180.0) {
lapHue = abs(modHue - 360.0);
} else {
lapHue = modHue;
}
// saturation difference
float satC = saturation(_img.pixels[pixIndex]);
float satN = saturation(_img.pixels[pixIndex - _img.width * chkDiv]);
float satNW = saturation(_img.pixels[pixIndex - _img.width * chkDiv - chkDiv]);
float satNE = saturation(_img.pixels[pixIndex - _img.width * chkDiv + chkDiv]);
float satW = saturation(_img.pixels[pixIndex - chkDiv]);
float satE = saturation(_img.pixels[pixIndex + chkDiv]);
float satS = saturation(_img.pixels[pixIndex + _img.width * chkDiv]);
float satSW = saturation(_img.pixels[pixIndex + _img.width * chkDiv - chkDiv]);
float satSE = saturation(_img.pixels[pixIndex + _img.width * chkDiv + chkDiv]);
float lapSat = sqrt(
pow(
- satW * 2.0
- satNW
- satSW
+ satE * 2.0
+ satNE
+ satSE
, 2) +
pow(
+ satN * 2.0
+ satNW
+ satNE
- satS * 2.0
- satSW
- satSE
, 2)
);
// brightness difference
float briC = brightness(_img.pixels[pixIndex]);
float briN = brightness(_img.pixels[pixIndex - _img.width * chkDiv]);
float briNW = brightness(_img.pixels[pixIndex - _img.width * chkDiv - chkDiv]);
float briNE = brightness(_img.pixels[pixIndex - _img.width * chkDiv + chkDiv]);
float briW = brightness(_img.pixels[pixIndex - chkDiv]);
float briE = brightness(_img.pixels[pixIndex + chkDiv]);
float briS = brightness(_img.pixels[pixIndex + _img.width * chkDiv]);
float briSW = brightness(_img.pixels[pixIndex + _img.width * chkDiv - chkDiv]);
float briSE = brightness(_img.pixels[pixIndex + _img.width * chkDiv + chkDiv]);
float lapBri = sqrt(
pow(
- briW * 2.0
- briNW
- briSW
+ briE * 2.0
+ briNE
+ briSE
, 2) +
pow(
+ briN * 2.0
+ briNW
+ briNE
- briS * 2.0
- briSW
- briSE
, 2)
);
if (
// hue difference
lapHue > 60.0
&& satC > 10.0
&& briC > 30.0
) {
edges.add(new PVector(idxW, idxH));
} else if (
// saturation difference
lapSat > 50.0
&& satC > 10.0
&& briC > 30.0
) {
edges.add(new PVector(idxW, idxH));
} else if (
// brightness difference
lapBri > 50.0
&& briC > 10.0
&& briC < 80.0
) {
edges.add(new PVector(idxW, idxH));
}
}
}
int removeCnt = edges.size() - _edgeNum;
for (int i = 0; i < removeCnt; i++) {
edges.remove(floor(random(edges.size())));
}
return edges;
}
/**
* typicalHue : detect typical hue value.
* @param _img : pick up hue value from this image.
* @param _pvs : coordinates in _img.
* @return hue value.
*/
private float typicalHue(PImage _img, ArrayList<PVector> _pvs) {
int[] cnt = new int[12];
for (int i : cnt) {
i = 0;
}
for (PVector p : _pvs) {
int pIdx = floor(p.y * _img.width + p.x);
cnt[floor(hue(_img.pixels[pIdx]) / 30.0)]++;
}
int hueValue = 0;
int maxNum = 0;
for (int i = 0; i < 12; i++) {
if (cnt[i] > maxNum) {
hueValue = i;
maxNum = cnt[i];
}
}
return hueValue * 30.0;
}
/*
Copyright (C) 2020- deconbatch
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 3 of the License, or
(at your option) any later version.
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU General Public License for more details.
You should have received a copy of the GNU General Public License
along with this program. If not, see <http://www.gnu.org/licenses/>
*/
deconbatch
To make something is my pleasure. Having fun in endless trial-and-error. Now, I'm loving creative coding and showing some example code in my blog. Mainly creating Processing code examples.
Can this type of code be used without a picture? I am just learning all of this. And can the 3d be applied to p5.js? I tried to figure out how to apply similar effects to an existing P5.js code that I made. I know it is different from processing.
Maybe yes.
The similar work without a picture is here. Enjoy! 😀
https://www.deconbatch.com/2020/12/welcome-to-my-nightmare-creative-coding.html