replicate / all-mpnet-base-v2

This is a language model that can be used to obtain document embeddings suitable for downstream tasks like semantic search and clustering.

  • Public
  • 2.2M runs
  • T4
  • GitHub
  • Paper
  • License

Input

string
Shift + Return to add a new line

A single string to encode.

string
Shift + Return to add a new line

A JSON-formatted list of strings to encode.

Output

[ 0.02005307748913765, 0.041290685534477234, 0.003116735490038991, 0.030528543516993523, -0.02006707154214382, -0.05383807793259621, 0.027094805613160133, 0.0672181025147438, 0.012287364341318607, 0.021269099786877632, 0.0003240387886762619, -0.0055049145594239235, -0.021382303908467293, 0.012780381366610527, 0.022158289328217506, 0.12000644952058792, 0.07592634856700897, 0.003038960276171565, 0.014363300055265427, 0.018159734085202217, -0.029201511293649673, 0.03300856798887253, 0.008941196836531162, 0.03897031769156456, -0.0036313594318926334, -0.01403711922466755, 0.03223872184753418, -0.03555147722363472, 0.00040921237086877227, -0.013119419105350971, 0.011497718282043934, 0.0094462214037776, -0.020669840276241302, 0.06250236928462982, 0.000001808083766263735, -0.006570104975253344, -0.03161371499300003, -0.03246781602501869, -0.031467899680137634, 0.002422950230538845, 0.010975888930261135, 0.01982017792761326, 0.012429546564817429, -0.029842903837561607, -0.036032065749168396, 0.028200093656778336, 0.04754353687167168, -0.028942307457327843, 0.010988772846758366, 0.011579218320548534, 0.002793207298964262, -0.012955536134541035, -0.08704198151826859, -0.009310574270784855, 0.018174652010202408, -0.06601182371377945, 0.0229220911860466, 0.0013939406489953399, -0.017548425123095512, -0.0012619340559467673, -0.018179314211010933, 0.043069615960121155, -0.014246347360312939, 0.004269869066774845, -0.00905574019998312, 0.005105658434331417, 0.030704185366630554, -0.04042154550552368, -0.03436288982629776, 0.0029063536785542965, 0.024458711966872215, -0.028325600549578667, -0.011808802373707294, 0.012212825939059258, -0.03724754974246025, -0.0003263512044213712, 0.07950270175933838, -0.005977529101073742, -0.059339091181755066, -0.03371788561344147, 0.0015379991382360458, 0.06698190420866013, 0.02031983807682991, -0.0053976159542799, -0.05364103242754936, 0.08225660771131516, -0.02942628413438797, -0.00488048093393445, 0.01461239904165268, -0.0531054325401783, 0.012338352389633656, -0.010653212666511536, 0.012831579893827438, -0.021721649914979935, -0.010193764232099056, 0.02282085455954075, -0.007184533402323723, 0.03230898827314377, 0.0633610337972641, -0.13401064276695251, -0.09183210134506226, -0.00906356144696474, -0.02710827812552452, 0.033054981380701065, 0.00043601676588878036, -0.009935955516994, 0.01017686352133751, -0.02511931210756302, -0.052157312631607056, 0.045824550092220306, 0.0038466756232082844, -0.009671383537352085, 0.008460070937871933, 0.03739413619041443, -0.010988891124725342, -0.06011807918548584, -0.02695130743086338, 0.00701861409470439, 0.004124033264815807, 0.04288149252533913, 0.008134222589433193, 0.00026375471497885883, 0.016047179698944092, -0.036170244216918945, -0.04677179083228111, -0.04521261528134346, -0.02576344646513462, -0.014711554162204266, 0.007595192175358534, 0.00878892932087183, 0.025261975824832916, -0.03619908168911934, 0.01809735596179962, -0.04194597899913788, 0.02697337418794632, 0.0385948084294796, -0.019632568582892418, -0.09417718648910522, -0.006960928440093994, 0.031198110431432724, -0.05572546645998955, -0.046894606202840805, -0.011007367633283138, 0.0009745990391820669, -0.0531373992562294, 0.004567625001072884, -0.04325222969055176, -0.060625068843364716, 0.006686283741146326, 0.008777173236012459, -0.0035451448056846857, 0.038806263357400894, -0.013935470953583717, -0.012266289442777634, -0.019012833014130592, 0.02198869176208973, 0.07520777732133865, -0.0019348661880940199, -0.030113011598587036, 0.03329329192638397, 0.05692151561379433, 0.015118454582989216, -0.03195987269282341, 0.02859192155301571, -0.036991313099861145, -0.04123535752296448, 0.050108447670936584, 0.012634576298296452, 0.023254824802279472, -0.038963865488767624, -0.058212488889694214, 0.012361779808998108, 0.01262666191905737, 0.009765070863068104, 0.03082389570772648, 0.009568044915795326, 0.018287228420376778, -0.004171018488705158, -0.032876405864953995, 0.038755759596824646, -0.03931901603937149, -0.022298764437437057, -0.006356417667120695, -0.045731715857982635, -0.005370360799133778, 0.008975175209343433, 0.06607916206121445, -0.04161756858229637, -0.01052190363407135, -0.0057122292928397655, -0.02698146179318428, -0.0006865145987831056, -0.028618011623620987, -0.05200434476137161, 0.03994189202785492, -0.04606958106160164, -0.03582562506198883, 0.027195870876312256, 0.020086532458662987, 0.05118211731314659, -0.02854614146053791, -0.02500780113041401, 0.10322774201631546, 0.06445720046758652, 0.02186265029013157, -0.011607514694333076, 0.007793340366333723, -0.010812326334416866, -0.03160707652568817, 0.03646469861268997, 0.013908675871789455, 0.03307662904262543, 0.06152082607150078, 0.016911495476961136, 0.033215317875146866, 0.047714632004499435, 0.030191970989108086, 0.047714490443468094, -0.036467041820287704, 0.09032099694013596, -0.010330918245017529, -0.0361456573009491, -0.0023837233893573284, -0.02055894024670124, 0.0529734268784523, -0.03648071736097336, 0.029766960069537163, 0.020662350580096245, -0.03414151445031166, 0.007700630929321051, 0.024196114391088486, 0.007781180553138256, -0.022023880854249, -0.022077729925513268, -0.0036830962635576725, -0.0022357709240168333, -0.021306924521923065, 0.07636149972677231, -0.04444950073957443, -0.03391217440366745, -0.02635655552148819, -0.07593724876642227, 0.02588762901723385, 0.017808111384510994, 0.008260337635874748, 0.015746498480439186, 0.019916700199246407, -0.0448385514318943, 0.029292339459061623, -0.03971479460597038, 0.03187505155801773, 0.01831423118710518, 0.06184292957186699, -0.01847018674015999, 0.03871694952249527, -0.06277653574943542, -0.017914624884724617, 0.024785200133919716, 0.00396379642188549, -0.07886072248220444, 0.04860173538327217, -0.043387241661548615, 0.0015239713247865438, 0.00585314491763711, 0.038212671875953674, -0.018921954557299614, 0.023159336298704147, -0.020757043734192848, -0.02968326397240162, 0.011024790816009045, -0.003512849099934101, -0.05569140240550041, 0.011618576012551785, -0.007775192614644766, 0.004529906902462244, 0.011513316072523594, -0.010556856170296669, 0.04904318228363991, -0.028170345351099968, 0.022104114294052124, 0.011435477994382381, -0.062447141855955124, -0.03936198353767395, 0.0029518085066229105, 0.015941917896270752, -0.01292079035192728, 0.036533012986183167, -0.11661049723625183, -0.048633649945259094, -0.00980621762573719, -0.025697961449623108, -0.007944745942950249, 0.0168209969997406, -0.034729965031147, 0.04490415006875992, -0.023888353258371353, -0.03810105100274086, 0.03612058609724045, -0.002279156818985939, 0.015648432075977325, -0.03875335678458214, 0.023432621732354164, 0.04034322500228882, 0.005940825212746859, -0.05779925733804703, -0.021058339625597, 0.05357707664370537, 0.029143162071704865, -0.03001994453370571, 0.011188758537173271, -0.01968931034207344, 0.021110011264681816, -0.005704880226403475, -0.007694427855312824, 0.013872680254280567, 0.004046231973916292, -0.035468168556690216, 0.0256916843354702, -0.03445333614945412, 0.0696055144071579, 0.03513667732477188, -0.028299184516072273, 0.04760720580816269, -0.01827189140021801, 0.015504308044910431, 0.02149580977857113, -0.018035341054201126, 0.050754617899656296, 0.05283072590827942, -0.05100002884864807, -0.001782214269042015, 0.0004607501905411482, -0.05962703377008438, 0.020883122459053993, 0.00873387698084116, 0.02370195835828781, -0.03895427659153938, -0.039850905537605286, -0.013572745025157928, 0.030265754088759422, -0.00994892232120037, 0.01803990826010704, -0.01291158702224493, -0.0031367498449981213, -0.08617670834064484, 0.009814002551138401, 0.08505069464445114, 0.0047349343076348305, -0.015542026609182358, 0.01969694346189499, -0.05336088314652443, 0.008665483444929123, -0.005943557247519493, 0.021634135395288467, -0.062486667186021805, -0.04341517761349678, -0.03598435968160629, 0.0005188887007534504, 0.05333565175533295, -0.009435477666556835, -0.009548589587211609, -0.026048803701996803, -0.05691111087799072, -0.046096183359622955, 0.02461027354001999, -0.05264008045196533, 0.002043773652985692, 0.024180365726351738, 0.03463289886713028, -0.017350729554891586, -0.045313600450754166, 0.00958498939871788, -0.04916699230670929, 0.018840786069631577, -0.0019373050890862942, -0.015995655208826065, 0.021619610488414764, 0.06462422013282776, -0.03540575131773949, 0.0017355073941871524, 0.037755902856588364, -0.024441061541438103, -0.025701845064759254, -0.02128761075437069, -0.02081909216940403, 0.024059617891907692, -0.02623576484620571, -0.06459119915962219, -0.019465789198875427, -0.00735819386318326, -0.02194376103579998, 0.029248222708702087, -0.012648330070078373, 0.027415355667471886, -0.03001350164413452, 0.03371749073266983, 0.035457879304885864, -0.00224535190500319, -0.019163137301802635, -0.047207340598106384, -0.014938184060156345, -0.06184227392077446, 0.06361186504364014, 0.006591165438294411, -0.01952475681900978, -0.06891963630914688, 0.05312427133321762, 0.001559905125759542, -0.06789834797382355, -0.08975519984960556, -0.0529046356678009, -0.05181577056646347, -0.03094862774014473, -0.017606958746910095, 0.06855621188879013, -0.06800869852304459, -0.019925471395254135, 0.06157136708498001, -0.00226752576418221, 0.019192812964320183, 0.03551565483212471, -0.008440214209258556, -0.036066167056560516, -0.008386615663766861, -0.03447746858000755, 0.0646761953830719, 0.036002010107040405, 0.018704554066061974, 0.02490515075623989, -0.008851908147335052, -0.0306098535656929, -0.04591144621372223, -0.01495369989424944, 0.05319483205676079, 0.03356357291340828, 0.008316580206155777, 0.002120785880833864, -0.024488531053066254, -0.03297191113233566, -0.04471508413553238, 0.06341686844825745, -0.004676141310483217, 0.11745765060186386, -0.02496039867401123, 0.003492281073704362, 0.03044806979596615, -0.00704230135306716, -0.008182715624570847, -0.0514691062271595, 0.018217194825410843, 0.027681613340973854, -0.08006452769041061, 0.03389345854520798, 0.058281898498535156, 0.03764202073216438, -0.026281453669071198, -0.05734710022807121, 0.010255002416670322, 0.015076608397066593, -0.012368838302791119, 0.0009709211881272495, 0.053436096757650375, 0.0007464365335181355, -0.04201324284076691, 0.06596338003873825, 0.0527116060256958, -0.0022870004177093506, -0.09733686596155167, -0.013886315748095512, -0.007824396714568138, 0.03398645669221878, -0.006993491668254137, 0.022295193746685982, -0.0077706025913357735, -0.00018201953207608312, 0.015512525103986263, -0.01593499630689621, 0.05000387132167816, 0.025041891261935234, 0.04645547643303871, -0.028457311913371086, -0.03783227503299713, -0.005659719463437796, 0.04990801587700844, -0.01683698780834675, 0.018938127905130386, 0.06512930244207382, 0.01633528247475624, -0.025264360010623932, -0.0377030223608017, 0.0028604399412870407, -0.06386826187372208, -0.00919246207922697, 0.013814538717269897, -0.015444939956068993, 0.041040413081645966, 0.00006619590567424893, -0.004870264790952206, -0.010712578892707825, -0.007292512338608503, 0.014273833483457565, -0.015574409626424313, 0.02192522957921028, 0.00018357363296672702, -0.020581543445587158, 0.00030544729088433087, 0.03308060020208359, 0.013934546150267124, 0.022513583302497864, -0.01869012415409088, -0.02165648341178894, 0.05089182406663895, 0.03467390686273575, -0.03420324996113777, -0.01309574581682682, 0.062211330980062485, -0.051981572061777115, -0.025703562423586845, 0.01753002405166626, -0.05144372954964638, 0.04831225797533989, -0.017150787636637688, 0.021171020343899727, 0.016288986429572105, -0.02100219950079918, 0.002770144259557128, 0.001608960097655654, -0.031415410339832306, 0.023885449394583702, 0.006652317009866238, 0.03861752152442932, -0.04789160564541817, -0.08788925409317017, 0.022040145471692085, -0.04013596102595329, -0.015888426452875137, 0.028185106813907623, 0.004646685440093279, 0.0014927230076864362, 0.028305374085903168, -0.06806475669145584, -0.022566281259059906, 0.023827841505408287, 0.020081602036952972, -0.008055934682488441, -0.03161204233765602, -0.0009819816332310438, 0.09394478797912598, 0.07452372461557388, -0.01616237685084343, -0.022954082116484642, 0.028225334361195564, -0.044956233352422714, 0.009193492121994495, -0.04668552055954933, -6.81819416913559e-33, -0.018732745200395584, -0.04261036589741707, 0.006103006657212973, 0.030934680253267288, 0.03885092958807945, 0.020690426230430603, -0.02763793244957924, 0.034022875130176544, -0.018702054396271706, -0.025176720693707466, 0.014377639628946781, 0.058167632669210434, 0.04948727414011955, 0.024207953363656998, 0.04419352859258652, -0.007449750322848558, 0.02005937695503235, 0.03853359445929527, 0.002096405951306224, -0.043226458132267, 0.006192795932292938, 0.016593005508184433, 0.10367501527070999, 0.04038487747311592, 0.03730115294456482, -0.012070246040821075, -0.07586873322725296, 0.03934392333030701, 0.012630434706807137, -0.039407067000865936, -0.013522635214030743, -0.01971348002552986, -0.020103096961975098, -0.055199362337589264, 0.041721049696207047, -0.008434473536908627, 0.03450300544500351, 0.011688840575516224, -0.003980678506195545, 0.042574841529130936, -0.00084398448234424, -0.03019452467560768, 0.005795097444206476, -0.024073265492916107, -0.028235062956809998, 0.04092521220445633, -0.00707671046257019, -0.008575071580708027, -0.013012491166591644, -0.02017991431057453, -0.018889369443058968, 0.031238239258527756, -0.0015596980229020119, 0.006149514112621546, 0.02365478128194809, 0.005150124430656433, -0.011567061766982079, -0.034206703305244446, -0.06389646977186203, -0.04468664899468422, -0.0025598544161766768, 0.03616861626505852, 0.04615003988146782, 0.019468439742922783, 0.0029902805108577013, 0.04821424186229706, 0.015107547864317894, 0.026465563103556633, 0.07379564642906189, 0.018050337210297585, -0.011061164550483227, 0.10132238268852234, -0.06350847333669662, -0.00010986173583660275, -0.014565098099410534, -0.0708066076040268, 0.05043400451540947, -0.006725891958922148, 0.006638015154749155, 0.0016703561414033175, 0.07089701294898987, -0.013210994191467762, 0.013233928941190243, -0.024296579882502556, 0.02771163359284401, -0.0014026304706931114, -0.02692767232656479, 0.05222095176577568, 0.023656455799937248, 0.0226877611130476, 0.021771889179944992, 0.02571282908320427, -0.01822366937994957, 0.010636121034622192, 0.03246661275625229, -0.07338036596775055, -0.00034971607965417206, 0.015259853564202785, -0.008504532277584076, 0.019270632416009903, -0.011139986105263233, -0.0058557260781526566, -0.08614062517881393, 0.04375318065285683, -0.0040136659517884254, -0.014016504399478436, 0.009164569899439812, -0.015288309194147587, -0.051065780222415924, 0.010592288337647915, 0.008534874767065048, 0.010640167631208897, 0.01608237251639366, -0.08793109655380249, 0.004160149954259396, -0.047099851071834564, 0.0023518649395555258, 0.0017590761417523026, 0.04262324050068855, 0.00893627293407917, 0.061083581298589706, 0.023118427023291588, -0.034400951117277145, 0.049451716244220734, -0.0648464560508728, 0.009903691709041595, 0.057059403508901596, 0.01879001036286354, 0.0334613136947155, 0.03429855778813362, 0.025084925815463066, 0.0018754827324301004, 2.4177288082682935e-7, 0.017704587429761887, 0.020610129460692406, -0.016728322952985764, 0.04256327077746391, 0.0031685265712440014, 0.04412764683365822, 0.04905141890048981, -0.010343044996261597, -0.06658416241407394, -0.11236967146396637, -0.004309413023293018, -0.01889633759856224, 0.02235846035182476, -0.013629970140755177, -0.0583941675722599, 0.027926890179514885, -0.016662467271089554, 0.011380606330931187, -0.05874524638056755, -0.0560377761721611, -0.023430993780493736, 0.046660277992486954, 0.036648623645305634, -0.02544967085123062, -0.00401556259021163, -0.08552922308444977, 0.008331958204507828, -0.03432827070355415, 0.0034007166977971792, 0.005714101251214743, 0.012897246517241001, -0.05649283528327942, -0.003663072595372796, -0.04178640991449356, -0.03229319676756859, 0.04049057140946388, 0.08136294782161713, 0.0750039741396904, 0.03683619573712349, 0.02858181670308113, 0.0021638120524585247, -0.10193269699811935, 0.046192679554224014, 0.044849544763565063, 0.0366038978099823, 0.033597420901060104, -0.04933731257915497, -0.012788296677172184, -0.009804074652493, 0.008237485773861408, 0.001833329675719142, 0.016655534505844116, -0.028216155245900154, -0.0017009192379191518, -0.003609209321439266, -0.01718311198055744, 0.031048981472849846, 0.01821393519639969, 0.05766703188419342, 0.018518393859267235, -0.029903125017881393, 0.008654993027448654, -0.03128441050648689, 0.002452449407428503, 0.0001696505642030388, -0.006398066878318787, 0.011106213554739952, 1.2324123842198277e-34, 0.011906014755368233, -0.03810819983482361, -0.0634765774011612, 0.018063949421048164, 0.04244471713900566, -0.008074632845818996, 0.0008796924375928938, 0.021606260910630226, 0.024893855676054955, -0.009295704774558544, 0.00788271613419056 ]
Generated in

Run time and cost

This model costs approximately $0.00022 to run on Replicate, or 4545 runs per $1, but this varies depending on your inputs. It is also open source and you can run it on your own computer with Docker.

This model runs on Nvidia T4 GPU hardware. Predictions typically complete within 1 seconds.

Readme

This model is a sentence-transformer based on MPNet, an encoder-style language model introduced by Microsoft.

You can use this model to support downstream tasks like document clustering and semantic search.

Model description

This model obtains token-level embeddings and then aggregates them with mean-pooling to produce a single 768-dimensional document embedding.

The base model has been fine-tuned with a contrastive learning objective to predict sentence pairs using a dataset of 1 billion sentence pairs.

Intended use

The fine-tuning procedure for this model makes it particularly well suited for encoding the semantic content of short documents, such as sentences. The maximum sequence length of the fine-tuning data was 384 tokens. This means that representational fidelity may degrade as sequence length exceeds this threshold.

However, this issue can be mitigated with additional fine-tuning on relevant data.

Ethical considerations

Language models are widely-known to encode social biases and it should be assumed that this model is not an exception.